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Abstract 

Over  the  past  decade,  a  considerable  amount  of  attention  has  been  given  to  federal 
legislation  in  making  the  federal  government  operate  more  efficiently  and  effectively  by 
concentrating  on  Information  Resource  Management  and  Information  Technology.  In 
1996,  the  Clinger-Cohen  Act  was  passed,  creating  the  role  of  Chief  Information  Officer 
(CIO)  in  each  agency  of  the  Federal  Government. 

This  study  assesses  the  impacts  of  the  Clinger-Cohen  Act  (CCA)  on  the  USAF, 
with  an  emphasis  on  the  Federal  CIO  Council  Core  Competencies.  Given  that  this  law 
and  supporting  policies  have  been  in  place  for  nearly  six  years,  it  would  be  valuable  to 
assess  USAF  eomplianee  with  CCA  as  well  as  its  impaet  on  the  USAF. 

It  is  intended  that  the  information  gathered  may  help  the  USAF  to  be  a  better 
steward  of  the  nations  eritieal  information  and  finaneial  resources,  and  to  better  provide 
eritieal  information  eapabilities  to  the  warfighter,  thus  ensuring  information  superiority 
over  our  nations  adversaries. 

Findings  of  this  survey  provide  evidenee  that  the  USAF  is  in  compliance  with  the 
CCA,  and  identifies  impaets  of  the  CCA  on  the  USAF.  Among  these  impacts  are:  IRM 
responsiveness  has  eontributed  to  mission  aeeomplishment.  Strategic  planning  includes 
information  as  well  as  IT,  teehnology  has  improved  efficiency,  baseline  performance 
measures  are  more  realistie,  alignment  of  organizational  structure  improves  critical 
services,  and  standardization  of  poliey  and  processes  optimizes  IT  resources. 
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SIX  YEARS  AFTER  ITS  PASSAGE,  WHAT  HAS  BEEN  THE  IMPACT  OF  THE 


CLINGER-COHEN  ACT  ON  THE  U.S.  AIR  FORCE? 


I.  Introduction 


Overview 

The  Clinger-Cohen  Act  (P.L.  104-106)  was  signed  into  federal  law  on  10 
February  1996  and  became  effective  on  8  August  1996.  Federal  Chief  Information 
Officers  (CIO)  were  mandated  in  every  (23)  executive  branch  agency  as  a  result.  Not 
long  after,  President  Clinton  signed  Executive  Order  13011  (1996),  “Federal  Information 
Technology,”  creating  the  Federal  CIO  Council  (CIOC)  and  outlining  additional  details 
concerning  the  responsibilities  and  duties  of  these  same  CIOs.  Generally  speaking,  the 
creation  of  the  CIO  position  was  a  strategy  for  federal  government  leadership  to  manage 
out-of-control  federal  spending  on  Information  Technology  (IT).  Likewise,  the 
establishment  of  the  CIOC  represented  the  president’s  effort  to  improve  federal  IT 
management  by  building  an  inter-agency  forum  that  was  chartered  to  improve  agency 
practices  on  such  matters  as  the  design,  modernization,  use,  sharing,  and  performance  of 
agency  information  resources  (EO  13011,  1996). 

Five  years  later,  what  has  been  the  impact  of  this  legislation?  This  study  will 
assess  the  effects  of  the  office  of  the  AF-CIO  on  the  USAF  as  an  organization,  based  on 
the  requirements  of  the  CCA  and  direction  from  the  Federal  CIOC.  Given  that  these 
federal  policies  have  been  in  place  for  over  five  years,  it  would  be  valuable  to  assess  the 
outcomes  of  the  CCA  and  the  office  of  the  AF-CIO.  It  will  describe  the  association  of 
the  office  of  the  AF-CIO  to  the  CIO  provisions  of  the  CCA  and  CIOC  guidance.  The 
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study  will  examine  and  relate  relevant  CIO  issues  from  the  private  sector  and  public 
sector.  Finally,  this  research  will  assess  the  effects  of  the  CCA  on  the  USAF  and  USAF 
operations. 

Background 

The  CCA  is  the  law  that  created  federal  CIO  positions  and  is  quoted  by  the 
Administration,  Congress,  and  government  agencies  as  the  authoritative  source  on  how 
this  Executive  Level  position  should  manage  and  perform  in  its  roles  and  responsibilities 
(GAO  2000a,  2000b).  Further,  the  CCA  led  to  the  creation  of  the  Federal  CIOC,  which 
has  raised  the  visibility  of  agency  CIOs,  as  the  Council  has  become  the  preeminent  inter¬ 
agency  IT  coordination  body  (EO  13011,  1996;  GAO,  2001).  The  CCA  also  changed 
the  way  that  agency  IT  systems  are  acquired  by  shifting  control  from  the  General 
Services  Administration  (GSA)  to  the  individual  agencies. 

While  it  is  possible  to  identify  a  large  body  of  prescriptive  Information  Resource 
Management  (IRM)  literature  (Deardon,  1987;  Boynton,  Jacobs,  and  Zmud,  1992; 
Markus  and  Keil,  1994;  Rockart,  Earl,  and  Ross,  1996;  Markus  and  Benjamin,  1997; 
Broadbent  and  Weill,  1997;  Feeny  and  Willcocks,  1998),  relatively  few  studies 
specifically  judge  whether  assessments  of  IRM  prescriptions  consistently  correlates  with 
technical  or  substantive  policy  compliance,  improved  IT  performance,  or  consequential 
affects  (Boynton,  et  ah,  1994;  Sambamurthy  and  Zmud,  1992;  Van  Shaik,  1985). 

Additionally,  several  government  studies  have  been  undertaken  on  the  CCA  and 
CIOs,  including  two  GAO  reports  on  federal  CIO  organizations  and  practices  (GAO, 
2000a,  2000b).  They  suggest  assessments  of  the  AF-CIO  office  would  be  effective  in 
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determining  CCA  compliance,  as  well  as  suggest  performance  evaluation  criteria  that 
would  assist  in  that  determination.  Numerous  studies  and  reports  have  prescribed  CIO 
roles  and  responsibilities,  accountability,  and  performance  effectiveness  as  well  as 
predicted  CIO  challenges  regarding  performance  and  technology.  While  prescriptive  IT 
management  literature  might  assess  technical  compliance  with  the  CCA,  this  study  is 
interested  in  surveying  substantive  compliance,  effects,  and  outcomes  of  the  CCA  and  the 
AF-CIO  office  on  the  USAF.  No  study  was  found  that  assesses  the  CCA  in  terms  if  its 
impact  on  the  USAF. 


Problem  Statement 

It  has  been  over  five  years  since  Congress  passed  the  Clinger-Cohen  Act,  the 
agency  CIO  position  was  created,  and  the  Federal  CIO  Council  was  established.  Five 
major  questions  could  be  asked  regarding  the  outcomes  of  this  legislation  and  it’s  impact 
on  the  USAF: 

1)  Are  the  USAF  implementation  ejforts  consistent  with  the  intent  of  the  CCA? 

2)  To  what  extent  can  a  model  of  the  AF-CIO  position  illustrate  the  level  of  the 
AF-CIO  office  implementation  efforts? 

3)  What  outcomes  can  be  found  as  a  result  of  the  CCA  and  the  AF-CIO  office 
requisite  to; 

A)  The  Federal  CIO  Core  Competencies, 

B)  The  performance  based  aspects  of  the  CCA,  Section  5123, 

C)  Key  IRM  achievement  areas  from  the  public  and  the  private  sector, 

D)  The  roles  and  responsibilities  of  the  AF-CIO 

4)  Have  changes  in  USAF  operations  occurred  as  a  result  of  instituting  an  AF-CIO 
office? 

5)  How  have  these  changes  impacted  the  USAF? 
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This  exploratory  study  will  propose  an  assessment  using  criteria  defined  in  the 
CCA,  prescriptive  literature  of  public  and  private  organizations,  and  the  results  of 
qualitative  research  measures.  The  resultant  assessment  collected  data  designed  to 
explicate  the  effect  of  the  CCA  and  the  AF-CIO  office  on  USAF  operations  and  the 
USAF.  This  study  suggests  the  results  may  be  generalizable  to  other  branches  of  military 
service  in  the  DoD,  and  possibly  to  other  federal  agencies. 

To  change  and  to  improve  are  two  different  things.  —  German  proverb 

Research  Focus 

The  intent  of  this  research  is  to  identify  the  impacts  the  CCA  has  made  on  the 
USAF.  The  researeh  will  eover  multiple  aspeets  of  the  agency  CIOs,  as  well  as  study 
how  the  offiee  of  the  CIO  has  been  implemented  in  the  USAF,  and  how  leadership  has 
helped  influenee  the  outeome  of  the  AF-ClO  offiee  on  USAF  operations.  Finally,  the 
research  will  discuss  the  impaets  of  offiee  of  the  AF-CIO  on  the  USAF. 

An  assumption  in  doing  this  research  is  that  USAF  compliance  with  the  CIO- 
related  mandates  of  the  CCA  is  in  the  best  interest  of  the  USAF.  By  this  1  mean  that  if 
the  USAF  creates  its  CIO  position,  roles,  and  responsibilities  in  accordance  with  the 
CCA,  some  benefit  to  mission  accomplishment  will  result. 

To  provide  an  interpretation  of  the  CCA  legislative  intent  with  respect  to  the 
federal  CIO  position,  the  research  will  examine  the  CCA,  other  federal  IRM  legislation, 
and  research  related  to  the  CIO  position.  Chapter  two  will  explore  the  research  that  has 
addressed  the  issue  of  the  CCA  and  explores  CIO  roles  and  responsibilities.  Based  on  the 
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results  of  the  literature  review,  an  assessment  of  AF-CIO  impacts  on  the  USAF  as  a  result 
of  the  CCA  will  be  addressed.  Chapter  three  will  address  the  methodology  for 
conducting  the  research.  Chapter  four  will  provide  the  results  of  the  data  collection. 
Chapter  five  will  discuss  the  results  obtained  in  Chapter  four  along  with  implications, 
limitations  of  the  study,  and  suggestions  for  future  research. 

Advantage  to  the  U.S.  Air  Force 

The  USAF  has  invested  much  time,  money,  and  energy  implementing  the 
mandates  of  the  Clinger-Cohen  Act.  The  value  of  implementing  these  mandates  range 
considerably  from  technical  compliance  with  federal  legislation,  to  the  substantive 
optimization  of  finite  information  resources  such  as  money,  time,  personnel,  and 
equipment.  Other  substantial  impaets  inelude  the  eapitalization  affects  of  strategic 
planning,  standard  systems,  standardized  IT  aequisition  and  technical  and  fiscal 
processes,  and  standard  performanee-based  measures.  Meeting  the  requirements  of  the 
CCA  could  support  the  leveraging  of  information  resources  toward  effective  operations 
and  eventually  the  USAF  Core  Competency  of  Information  Superiority.  Given  that  the 
CCA  mandates  have  been  in  existence  for  nearly  six  years,  it  may  be  beneficial  to  better 
understand  both  the  impact  that  this  legislation  has  had  on  the  USAF  and  its  implications 
for  future  operations. 
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11.  Literature  Review 


Overview 

In  qualitative  research,  literature  is  often  used  sparingly  and  inductively  to  frame 
the  research  problem  (Creswell,  1994).  However,  others  believe  that  a  broader 
theoretical  framework  is  required  to  focus  data-collection  efforts  and  reduce  the  chances 
of  the  researcher  being  inundated  with  large  amounts  of  data  (Miles,  et  al,  1994;  Yin, 
1994).  The  researcher  adopted  the  latter  view  to  employ  a  broad  observation  of  the 
literature  in  order  to  develop  the  research  focus  introduced  in  Chapter  1,  and  to  guide  the 
data  collection  and  analysis. 

In  order  to  conduct  an  assessment  of  the  impacts  of  the  Clinger  -Cohen  Act  of 
1996  (CCA)  on  the  Air  Force,  it  is  necessary  to  define  and  review  the  Information 
Resource  management  (IRM)  field  of  research,  federal  IT  policy  leading  up  to  the 
passage  of  the  CCA,  the  goals  of  the  CCA,  and  how  CIO  roles  and  responsibilities  have 
developed  to  implement  the  CCA.  It  is  also  necessary  to  place  into  context  the 
theoretical  model  in  which  to  base  the  assessment,  and  to  explain  the  results  in  terms  of 
outcomes  (or  impacts)  on  the  USAF  and  USAF  operations.  A  previous  study  regarding 
the  impacts  of  the  CCA  and  the  office  of  the  AF-CIO  on  the  USAF  could  not  be  found  by 
the  researcher. 

This  chapter  reports  a  review  of  the  literature  to  learn  what  has  been  published 
about  CIOs  and  the  management  of  information.  The  chapter  begins  with  a  review  of  the 
field  of  interest  and  the  historical  background  creating  the  agency  CIOs  in  the  federal 
government.  To  provide  an  understanding  of  the  CCA's  legislative  intent  with  respect  to 
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the  federal  CIO  position,  the  CCA  was  reviewed  along  with  other  federal  IRM  policy  and 
other  related  guidance.  Information  relating  the  CCA  and  Executive  Order  13011  is  also 
presented.  The  federal  policies  are  connected  to  the  AF-CIO  focus  areas  and  are 
examined  along  with  experiences  of  federal  and  state  government  agencies,  and  private 
sector  organizations.  This  section  summarizes  a  high-level  view  of  CIO  Core 
Competencies,  or  focus  areas,  of  government  CIOs  private  sector  CIOs,  and  finally  the 
AF-CIO.  Further,  the  relationship  between  the  roles  and  responsibilities  of  CIOs  are 
associated  with  an  organizational  context  and  policy  framework  that  makes  the 
relationship  more  understandable. 

Field  of  Interest 

Information  Resource  Management  (IRM)  is  the  field  of  interest  for  this  study. 
The  result  of  effective  IRM  by  the  office  of  the  AF-CIO,  in  the  context  of  federally 
mandated  policy  and  other  managerial  and  organizational  factors,  is  the  subject  of 
interest.  Although  the  terms  “information  systems”  (IS)  management,  “information 
management”  (IM),  and  “information  technology”  (IT)  management,  have  been  used 
interchangeably  in  literature,  (Boaden  and  Locket,  1991)  the  IRM  term  is  appropriate  for 
this  study  as  it,  “entails  a  broader  conceptual  definition  of  management  as  well  as  the 
human  resources  and  technical  components  more  typically  associated  with  IT 
management  (Lewin  and  Sprehe,  1996).  The  literature  also  expresses  concern  that 
existing  policies  emphasize  managing  technologies  at  the  expense  of  managing 
information  even  though,  “to  some  extent,  IM  issues  become  defined  as  IT  management 
issues  (Lewin,  et  ah,  1996). 
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The  study  uses  the  term  IT  management,  in  its  general  sense.  It  is  consistent  with 
the  definition  of  IRM  used  in  section  3502  (7)  of  title  44,  United  States  Code  as  “the 
process  of  managing  information  resources  to  accomplish  agency  missions  and  improve 
agency  performance,  including  through  reduction  if  information  collection  burdens  on  the 
public”  (United  States  Congress,  Title  44,  1997;  Section  3502). 

The  scope  of  this  research  effort  will  be  limited  to  identifying  and  reviewing 
federal  government  and  USAF  IT  policy  and  existing  IRM  theory  and  practice,  as 
captured  in  the  CCA,  and  its  impact  on  USAF  operations.  The  intent  of  the  research  is  to 
provide  an  assessment  of  progress  of  the  AF-CIO  office  and  USAF  operations  in 
accomplishing  the  objectives  of  the  CCA. 

Federal  IRM  Problems 

In  1994,  Senator  William  Cohen  of  Maine,  released  a  critical  report  on  federal  IT 
management  entitled,  "'Computer  Chaos:  Billions  Wasted  Buying  Federal  Computer 
Systems.”  In  the  report,  Senator  Cohen  identified  major  IT  projects  wasting  billions  of 
dollars  because  of  poor  management.  To  improve  the  success  of  IT  projects  in  the  public 
sector.  Senator  Cohen  stated  the  government  needs  to  do  better  planning  of  IT  projects 
particularly  when  they  define  objectives,  analyze  alternatives  and  establish  performance 
measures  that  link  to  agency  accomplishments  (Cohen,  1994).  Additionally,  Senator 
Cohen  declared,  “The  federal  government  is  the  largest  information  manager  in  the 
world. .  .The  ability  of  the  government  to  manage  this  information  has  a  profound  effect 
on  the  daily  lives  of  all  of  us. .  .Government  information  systems  are  headed  for 
catastrophic  failure  if  we  fail  to  address  the  challenge  of  modernization”  (Cohen,  1994). 
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Prior  to  Senator  Cohen’s  report,  the  General  Accounting  Office  (GAO) 
summarized  the  problems  identified  in  audits  (see  Table  1)  associated  with  federal  IT 
management  over  a  period  of  nearly  three  years  (General  Accounting  Office,  1992). 


Table  1:  Number  of  Reports  Citing  IT  Management  Problems 


Instances 

Problem 

66 

Inadequate  management  of  information  systems  development  lifecycle 

29 

Ineffective  oversight  and  control  of  information  resources  management  (IRM) 

16 

Inability  to  ensure  security  and  integrity  or  reliability  of  information  systems 

14 

Inabilityof  systems  to  work  together 

9 

Inadequate  resources  to  accomplish  IRM  goals 

22 

Cost  overruns 

20 

Schedule  delays 

7 

Systems  not  performing  as  intended 

18 

Inaccurate,  unreliable  or  incomplete  data 

8 

Systems  that  make  data  access  time  consuming  or  cumbersome 

Source:  GAO  Report  GAO/IMTEC-92-13FS  (General  Accounting  Office,  1992) 


Cohen’s  report  emphasized  these  failures  and  declared  that  government  efforts  to 
replace  its  antiquated  computer  systems  met  with  little  success  because  of  poor 
management,  inadequate  planning,  and  an  acquisition  process  that  is  too  cumbersome  to 
competitively  purchase  computer  technology  before  it  is  obsolete  (Cohen,  1994). 

Because  the  process  of  acquiring  federal  computer  systems  takes  longer  than  developing 
new  technology,  the  likelihood  is  increased  that  technology  will  be  obsolete  once 
delivered  (Ibid.).  The  report  cited  several  modernization  efforts  that  have  failed,  and 
faults  the  protest  process  as  a  major  factor  in  the  long  delays  in  acquiring  computers,  as 
well  as  higher  costs  due  to  court  and  personnel  costs.  Cohen’s  suggestions  to  improve 
the  situation  included  early  oversight  and  planning,  encouraging  innovation  through  pilot 
programs  using  new  procurement  ideas,  and  creating  incentives  for  both  the  government 
and  contractors  to  perform,  such  as  using  past  performance  as  an  evaluation  criteria. 
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To  meet  the  challenge,  numerous  changes  to  the  IRM  policy  framework  occurred 
during  the  1980’s  and  1990’s  through  legislation,  Office  of  Management  and  Budget 
(0MB)  directives,  and  IT  related  initiatives.  At  the  heart  of  these  federal  policies  are 
directives  to  control  IT  costs,  meet  IT  requirements,  and  create  a  strategy  to  measure  IT 
performance  and  effectiveness.  It  is  also  through  these  laws  and  policies  that  the  position 
of  Chief  Information  Officer  (CIO)  came  into  existence  in  federal  government  agencies, 
as  well  as  the  creation  of  a  Federal  CIO  Council. 

Federal  Legislation 

This  section  will  review  major  IT  legislation  and  policy  framework  related  to  IT 
management,  that  are  the  precursors  to  the  Clinger-Cohen  Act  of  1996. 

Brooks  Act  (P.L.  89-306) 

Under  the  Brooks  Aet,  the  Seeretary  of  Commerce  had  responsibility  for  federal 
computer  standards  through  the  National  Institute  of  Standards  and  Technology  (NIST). 
However,  0MB  retained  management  oversight  authority  (40  U.S.C.  Section  759 
(d)(5)(e)).  Additionally,  the  General  Services  Administration  (GSA)  was  charged  with 
the  authority  and  the  responsibility  to  acquire  and  operate  information  technology  as  well 
as  to  oversee  the  IT  acquisition  process  and  operation. 

In  comparing  the  Paperwork  Reduction  Act  of  1995  (PRA  ‘95)  IT  acquisition 

philosophy  to  that  of  the  1966  Brooks  Act,  John  Bertot  et  al.  stated: 

A  fundamental  objective  of  the  PRA  was  to  have  agencies  manage  their  own 
information  resources,  and  to  include  IT  as  part  of  this,  while  the  Brooks  Act’s 
primary  objective  was  for  agencies  to  acquire  IT  through  the  most  cost-effective  and 
efficient  means  (Bertot,  et  al,  1996). 
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Congressional  and  agency  criticism  of  the  Brooks  Act  was  that  its  emphasis  on 
cost-effectiveness  in  IT  acquisition  did  not  necessarily  produce  systems  that  allowed 
agencies  to  produce  efficient  information-based  services.  In  spite  of  this  criticism  of  the 
Brooks  Act,  PRA  '95  did  not  replace  or  modify  the  Brooks  Act  (Bertot,  et  al,  1996).  The 
CCA  repealed  the  1965  Brooks  Act,  which  was  characterized  as  “strict  regulatory  control 
over  IRM,  an  excessive  documentation  approval  process,  and  a  lengthy  acquisition  cycle 
in  which  systems  were  often  obsolete  when  finally  fielded”  (Johnson,  1997;  3). 

The  Paperwork  Reduction  Act  (PRA) 

In  1976,  President  Ford  established  the  Commission  on  Federal  Paperwork  to 
assess  the  impact  of  Government  reporting  requirements  on  businesses  and  individuals. 

In  1977,  the  Commission  launched  an  investigation  of  information  management 
practices.  The  Commission  observed; 

The  absence  of  a  body  of  doetrine  covering  the  effective  and  efficient  management  of 
information  resources  has  fostered  overlap  and  duplication  in  both  the  administrative 
controls  over,  and  organizational  structures  which  manage  information  gathering, 
processing,  and  dissemination  (Holden,et  al,  1996). 

In  1980,  Congress  passed  the  Paperwork  Reduction  Act  incorporating  the 
Commission's  findings  to  recognize  information  as  a  valuable  and  manageable  resource, 
reduce  federal  agency  paperwork  burdens  on  the  public  and  indusfry  by  1 5  percent  by 
1982,  and  to  centralize  federal  information  policy  functions  into  the  Office  of 
Information  and  Regulatory  Affairs  (OIRA)  within  0MB  (Holden,  et  al,  1996). 

The  Paperwork  Reduction  Act  of  1980  (PRA  '80)  also  connected  industry  and 
government  best  practices  in  IRM,  including  mandates  for  each  agency  to: 

1)  Carry  out  information  management  activities  in  an  efficient,  economical  manner. 
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2)  Designate  a  senior  official  or  officials  to  carry  out  agency  responsibilities  under  the 
Act, 


3)  Inventory  major  information  systems  and  review,  periodically,  its  management 
activities, 

4)  Ensure  that  its  systems  do  not  overlap  each  other  or  duplicate  systems  of  other 
agencies, 

5)  Develop  procedures  for  assessing  the  paperwork  burden  of  its  collection  activities, 

6)  Ensure  that  each  information  collection  request  submitted  to  nine  or  fewer  persons 
that  it  is  not  subject  of  the  provisions  of  PRA  '80.  (P.L.  96-5 11). 

The  PRA  was  amended  again  in  1995  (PRA  ’95)  (P.L.  104-13).  Congress 

intended  PRA  ‘95  to  strengthen  federal  IRM  and  concluded  that  the  IRM  concept  under 

the  PRA  was  not  flawed  but,  “rather  the  need  is  to  develop  an  improved  strategy  by 

which  to  apply  IRM”  (U.S.  Senate,  1994).  The  PRA  established  a  broad  mandate  for 

agencies  to  perform  IRM  activities  in  an  efficient,  effective,  and  economical  manner. 

To  assist  agencies  in  an  integrated  approach  to  information  resources  management, 
the  PRA  requires  that  the  Direetor  of  0MB  develop  and  implement  uniform  and 
consistent  information  resources  management  policies;  oversee  the  development  and 
promote  the  use  of  information  management  principles,  standards,  and  guidelines; 
evaluate  agency  information  resources  management  practices  in  order  to  determine 
their  adequacy  and  efficiency;  and  determine  compliance  of  such  practices  with  the 
policies,  principles,  standards,  and  guidelines  promulgated  by  the  Director  (P.L.  104- 
13). 


Perhaps  congress  had  intended  to  align  agencies’  strategic  missions  and 
strengthen  0MB  oversight  processes.  However,  these  objectives  were  not  really  new, 
and  no  new  mechanisms  or  resources  were  provided  to  assist  the  agencies  in  achieving 
them  (Benchboard,  1996). 

Reconciling  PRA  and  the  Brooks  Act 

“Congress  never  reconciled  the  differences  between  the  PRA  and  the  Brooks  Act” 
(Holden,  1996).  The  main  objective  of  the  PRA  was  centered  around  agency 
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management  of  their  own  information  resources  to  provide  efficient  and  effective 
information  services  with  regard  to  stewardship  of  federal  government  resources.  The 
Brooks  Act  objective  was  for  agencies  to  acquire  IT  through  the  most  cost  effective  and 
efficient  means.  Holden  points  out  that  “cost  effective  and  efficient  acquisitions  do  not 
necessarily  generate  information  technologies  that  allow  agencies  to  produce  efficient 
and  effective  information-based  services”  (Ibid).  While  the  intent  of  these  policies 
shared  the  interest  of  federal  IRM,  it  appears  that  differing  interpretations  caused 
consternation,  leading  to  the  development  of  the  Clinger-Cohen  Act  of  1996,  ultimately 
repealing  the  Brooks  Act. 

Government  Performance  and  Results  Act  (GPRA)  (P.L.  103.62) 

In  1993  Congress  passed,  and  the  President  signed  into  law,  the  GPRA.  The 
broad  intent  of  the  legislation  was  to  enhance  the  effectiveness,  efficiency,  and 
accountability  of  government  programs  by  directing  federal  agencies  to  more  singularly 
focus  their  management  efforts  on  the  results  that  are  achieved,  and  away  from  such 
traditional  concerns  such  as  staffing  and  activity  levels.  Under  GPRA,  agencies  must  set 
goals,  measure  performance,  and  report  on  their  accomplishments. 

Under  the  GPRA,  agencies  develop  five-year  plans  that  include  mission 
statements,  agency  goals  and  associated  program  performance  plans,  “to  establish 
objective,  quantifiable,  and  measurable... performance  objectives. ..unless  authorized  to  be 
in  an  alternative  form...”  (P.L.  103-62,  Section  1 15(a)(3)).  This  comprehensive  initiative 
“impacts  IRM  and  IT  management  in  that  they  establish  the  strategic  planning  processes 
with  which  to  align  IT  planning,  and  the  objective  agency -performance  measures  to 
evaluate  IT  contributions”  (Bertot,  et  al,  1996). 
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Recently  President  George  W.  Bush  cited  the  GPRA  in  “The  Presidents 

Management  Agenda”,  August  2001,  which  provides  the  President's  strategy  for 

improving  the  management  and  performance  of  the  federal  government; 

In  1993,  Congress  enacted  the  Government  Performance  and  Results  Act  (GPRA)  to 
get  the  federal  government  to  focus  federal  programs  on  performance.  After  eight 
years  of  experience,  progress  toward  the  use  of  performance  information  for  program 
management  has  been  discouraging.  According  to  a  General  Accounting  Office 
(GAO)  survey  of  federal  managers,  agencies  may,  in  fact,  be  losing  ground  in  their 
efforts  to  building  organizational  cultures  that  support  a  focus  on  results  (The 
President’s  Management  Agenda,  2001). 

The  assumption  is  that  new  information  technologies  should  be  a  great  advantage 
in  bringing  about  successful  program  performance,  however.  President  Bush  alludes  to 
the  possibility  that  results  are  not  being  properly  associated  to  performance  in  federal  IT 
management. 

0MB  Circular  A-1 30 

In  1985,  0MB  issued  the  first  extensive  policy  for  federal  IRM  in  Circular  A- 130: 
The  Management  of  Federal  Information  Resources,  incorporated  PRA  '80,  other  federal 
guidance  and  govemmenfindustry  best  practices  (Office  of  Management  and  Budget, 
1996).  For  more  than  a  decade  PRA  '80  and  0MB  A- 130  remained  the  federal 
government's  primary  IRM  guidance.  Together  they  created  the  government's  version  of 
the  term  and  scope  for  IRM,  the  position  of  an  agency  Senior  Official  for  IRM,  and 
evolved  these  concepts  through  two  major  revisions  of  the  PRA  (in  1986  and  in  1995) 
and  two  revisions  of  0MB  A-130  (1996  and  2000)  (Holden,  et  al,  1996).  The  1995 
reauthorization  of  the  PRA  introduced  a  dramatic  new  approach  to  federal  IT  oversight 
by  creating  a  Senior  IRM  Official  in  each  federal  agency,  reinforcing  the  IRM/TQM 
principles  of  0MB  A-130  (Holden,  et  al,  1996). 
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0MB  Circular  A-130  was  revised  in  mid-1996  to  align  with  PRA'95,  but  was 
issued  before  the  mandates  of  the  newly-passed  CCA  could  be  incorporated.  Reeently, 
0MB  Cireular  A-130  was  revised  with  Transmittal  Memorandum  No.  4,  on  November 
30,  2000.  The  mandates  of  the  CCA  were  incorporated.  Memorandum  No.  4  establishes 
a  eomprehensive  approaeh  for  executive  agencies  to  improve  the  acquisition  and 
management  of  their  information  resources  by, 

1)  foeusing  information  resource  planning  to  support  federal  agency  strategic 
missions; 

2)  implementing  a  eapital  planning  and  investment  control  process  that  links  to 
budget  formulation  and  execution;  and 

3)  rethinking  and  restrueturing  the  way  the  federal  government  does  their  work 
before  investing  in  information  systems.  The  PRA  establishes  a  broad  mandate 
for  ageneies  to  perform  their  information  resources  management  aetivities  in  an 
effieient,  effeetive,  and  economical  manner  (0MB  Circular  A-130  (5),  2000). 

To  assist  agencies  in  an  integrated  approach  to  information  resources 

management,  the  PRA  requires  that  the  Director  of  0MB  develop  and  implement 

uniform  and  consistent  information  resources  management  policies. 

Clinger-Cohen  Act  of  1996 

Congress  was  not  convinced  that  GPRA  and  PRA  ‘95  would  result  in  a  correction 
of  identified  federal  IT  problems  (Holden,  et  al,  1996).  They  therefore  passed  the  CCA 
less  than  a  year  after  PRA'95  in  order  to  create  a  synergy  of  IT  acquisition  reform  and 
program  performance  reviews,  and  to  consolidate  IT  oversight  under  CIOs  (Bertot,  et  al, 
1996).  The  CCA  also  modified  the  Computer  Security  Act  of  1987  and  the  National 
Institute  of  Standards  and  Technology  Act,  and  eliminated  the  Federal  IRM 
Regulations(FlRMR),  which  were  absorbed  into  the  Federal  Acquisition  Regulations 
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(FAR)  (Bertot,  et  al,  1996).  In  so  doing,  the  CCA  in  early  1996  replaced  PRA'95  as  the 
predominant  piece  of  legislation  dealing  with  federal  IRM  policy  (Ibid.). 

The  Clinger-Cohen  Act  created  the  statutory  position  of  Chief  Information  Officer 
in  major  federal  government  agencies.  It  requires  the  Office  of  Management  and  Budget 
(0MB),  the  agencies,  and  the  Chief  Information  Officers  to  improve  information 
technology  practices.  It  requires  mission  and  program  driven  strategic  planning  for 
information  technology.  It  requires  senior  user  management  guidance  to  ensure 
information  technology  activities  align  with  agency  plans  and  operations.  It  requires 
regular  assessments  of  information  technology  skills  inventory,  skills  requirements,  and 
skills  development  programs.  In  short,  the  Clinger-Cohen  Act  requires  the  development 
of  an  effective  and  efficient,  mission-oriented,  user-oriented,  results-oriented  information 
technology  practice  in  each  and  every  federal  agency. 

The  purpose  of  the  Clinger-Cohen  Act  is  to  ensure  that  the  federal  investment  in 
information  technology  is  made  and  used  wisely.  The  law  was  designed  to  increase 
competition,  eliminate  burdensome  regulations,  and  help  the  Government  benefit  from 
efficient  private  sector  techniques  (Johnson,  1997). 

Section  5125  of  the  CCA  obligates  each  executive  agency  to  appoint  a  CIO  and 
establish  a  process  to  acquire  and  manage  IT  investments.  This  section  also  had  the 
effect  of  amending  the  Paperwork  Reduction  Act  (codified  as  Chapter  35  of  Title  44  of 
the  US  Code)  by  specifically  designating  the  agency  CIO  as  the  officer  responsible  for 
information  resources  management. 

The  CCA  applies  to  all  federal  executive  agencies  and  all  IT  system  domains.  It 
applies  to  and  combines  automated  information  systems  (AIS),  command,  control  (C2), 
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communications  (C3),  computer  (C4),  and  intelligence  (C41)  systems  and  embedded 
systems.  The  Office  of  the  Secretary  of  Defense  (OSD)  explains  that,  "Recent  guidance 
from  0MB  places  added  emphasis  on  managing  investments,  to  include  weapons 
systems"  (0MB  Memorandum,  1997). 

Executive  Order  13011 

To  implement  PRA  ’95  and  the  CCA,  President  Clinton  issued  Executive  Order 
(EO)  13011,  Federal  Information  Technology  (July  16,1996).  This  order  established  a 
government-wide.  Federal  CIO  council  to  serve  as  a  forum  to  share  ideas  and  make 
government-wide  recommendations.  The  order  also  established  the  Government 
Information  Technology  Services  Board  to  confirm  that  National  Performance  Review 
(NPR)  recommendations  are  carried  out.  Concerns  over  National  Information 
Infrastructure  security  issues  resulted  in  a  revision  of  0MB  Circular  A- 130.  FAR  Section 
39.001,  Acquisition  of  Information  Resources,  has  been  rewritten  to  reflect  the  CCA 
procurement  policies. 

Section  3  of  Executive  Order  1301 1  created  the  Federal  Chief  Information 
Officers  Council  (CIOC).  Essentially,  the  Federal  CIOC  is  responsible  to  act  as  the 
“principal  interagency  forum  to  improve  agency  practices  on  such  matters  as  the  design, 
modernization,  use,  sharing,  and  performance  of  agency  information  resources”  (EO 
13011,  1996).  Essentially,  the  Federal  CIOC  has  been  entrusted  to; 

1)  Develop  recommendations  for  federal  information  technology  management  policy, 
procedures,  and  standards; 

2)  Share  experiences,  ideas,  and  promising  practices,  including  work  process  redesign 
and  the  development  of  performance  measures,  to  improve  the  management  of 
information  resources; 
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3)  Identify  opportunities  and  recommendations  for  sponsoring  cooperation  in  using 
information  resources; 

4)  Review  and  address  the  hiring,  training,  classification,  and  professional 
development  needs  of  the  federal  government  personnel  with  respect  to  IRM; 

5)  Make  recommendations  and  provide  advice  to  appropriate  executive  agencies. 

6)  Seek  the  views  of  the  Chief  Financial  Officers  Council,  Government  Information 
Technology  Services  Board,  Information  Technology  Resources  Board,  Federal 
Procurement  Council,  industry,  academia,  and  State  and  local  governments  on 
matters  of  concern  to  the  Council  as  appropriate.  (EO  13011,  Section  3) 

Currently,  the  council  membership  is  comprised  of  the  Chair  of  the  CIO  Council, 
the  Deputy  Director  for  Management  for  the  0MB,  the  Vice  Chair,  elected  by  the 
membership  of  the  CIO  Council,  and  represented  by  at  least  28  federal  executive  agency 
CIO  offices.  Membership  also  includes  associates  of  other  federal,  state,  and  private 
sector  IM  professionals.  Recently,  the  CIO  Council  included  participation  from  state 
government  and  private  sector  representatives.  The  Council  recognized  that  by  including 
the  experience  of  other  organization  CIOs  they  could  capitalize  on  synergistic  and 
cooperative  teaming  effects  that  make  CIO  initiatives  successful  (Feeny,  D,  1997). 

The  Federal  CIO  Council  was  established  to  more  effectively  contend  with 
federal  government  IT  management  issues  (Feeny,  1997).  This  E.O.  13011  directs  a 
coordinated  approach  to  IT  management  that  builds  on  existing  IRM  structures  of 
successful  practices  observed  in  the  private  sector  as  well  as  in  federal  and  state 
government  agencies.  Since  its  creation,  the  council  has  drafted  guidance  on  several 
issues  of  great  concern  for  federal  IT  management  including  topics  such  as  IT  capital 
planning,  identifying  critical  skills  required  by  CIOs,  enterprise  interoperability,  and 
standarding  measures  of  costs  and  benefits  (Federal  CIO  Council,  1998a,  1998b,  1999a, 
1999b). 
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Significance  of  the  CCA 

As  stated  earlier,  the  CCA  is  the  predominant  piece  of  legislation  dealing  with 
federal  IRM  policy  (Bertot,  et  al,  1996).  The  CCA  defines  the  role  of  the  CIO,  designates 
the  CIO  as  the  senior  (IT)  official  in  each  agency,  and  asserts  the  general  responsibilities 
for  the  position.  The  CCA  also  specifies  duties  and  qualifications  for  CIOs.  Creation  of 
the  CIO  positions  for  organizations  gives  an  executive-level  focus  and  accountability  for 
IT  and  management  issues  within  agencies  that  are  meant  to  guide  a  greater  level  of 
accountability  for  delivering  effective  technology  systems  and  services  (GAO,  2000c). 

PRA  ’95  created  chief  IRM  executives  in  federal  government  agencies  and  was 
enacted  for  the  purpose  of  improving  the  management  of  information  resources  within 
the  exeeutive  braneh  of  government  (PRA,  1995).  The  ehanges  in  the  CCA  reflect  the 
experienees  of  PRA  legislation  and  events  both  internal  and  external  to  the  government. 

The  legislative  history  of  the  CCA  implies  there  were  at  least  three  major  areas  of 
eoneem;  1)  serious  defieieneies  in  major  federal  information  teehnology  acquisitions, 
and  2)  the  need  to  reengineer  eapital  planning  and  performanee  measurement,  and  3) 
notable  suecesses  of  the  private  seetor  CIOs  position.  This  section  addresses  these  areas, 
as  well  as  others  that  are  of  interest  to  the  researeher. 

IT  Acquisition  Changes 

By  focusing  on  IT  acquisition  and  management  of  large  development  projects,  the 
CCA  implies  that  large  system  development  problems  in  the  federal  government  could  be 
avoided  (Beachboard,  1996).  Similar  situations  may  occur  in  the  private  sector,  but 
differences  in  the  public/private  seetor  acquisition  goals  and  processes  may  reduce  the 
severity  and  occurrence.  Better  alignment  of  authority  and  responsibility  is  a  classic 


19 


management  approach  for  large,  complex  challenges,  both  within  and  outside  of  the  IT 
community  (Cohen,  1994). 

Aligning  the  authority  and  responsibility  for  large  system  development  projects  is 
clearly  one  of  the  tenets  of  the  private  sector's  movement  to  CIO  designations  (Cohen, 
1994).  The  clear  goal  is  to  make  the  CIO  part  of  the  agency  head's  governing  body.  By 
accomplishing  this,  the  federal  government  might  be  able  to  take  advantage  of  the 
benefits  and  successes  learned  by  the  private  sector  and  other  organizations. 

Business  Processes,  Capital  Planning,  and  Performance  Measurement 

The  CCA  recognizes  the  need  for  agencies  to  reassess  business  processes  and 
focus  on  capital  planning  and  performance-based  measurement.  This  legislation 
explicitly  requires  an  analysis  of  organizational  missions,  benchmarking,  and  a 
performance  assessment  of  business  processes.  Based  on  this  observation,  mission- 
related  and  administrative  processes  are  redesigned  prior  to  investing  significantly  in 
information  technology  (IT)  to  support  those  missions.  Simply  stated,  the  CCA  mandates 
that  agencies  must  maximize  the  potential  of  technology  to  improve  performance  rather 
than  simply  automating  inefficient  processes. 

The  CCA  requires  federal  agencies  to  integrate  IT  investment  plans  and 
performance  measures  into  the  budget  process.  CCA  Section  5123,  Performance  and 
Results  Based  Management,  can  be  found  in  Appendix  A.  Highlights  of  the  requirements 
include;  1)  establishing  a  process  to  select,  manage  and  evaluate  the  results  of  IT 
investments,  2)  submitting  annual  reports  on  progress  in  achieving  goals  with  budget 
submission,  3)  linking  IT  performance  measures  to  agency  programs,  and  4)  revising 
mission-related  processes  before  making  significant  IT  investments. 
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Basis  for  Federal  Agency  CIOs 

The  new  IT  management  provisions  in  the  CCA  offer  the  potential  to  improve 
how  government  agencies  decide  to  spend  money  on  IT  initiatives  (Beachboard,  1996). 
Built  on  practices  which  are  common  to  leading  public  and  private  organizations,  the 
CCA  assists  in  focusing  senior  management  attention  on  selecting  well  designed  projects 
with  sound  business  justifications  while  mitigating  risks  as  IT  investments  proceed 
through  development,  and  evaluating  actual  performance  improvement  results 
attributable  to  the  investments  (Beachboard,  1996).  A  common  understanding  of  the 
legislation's  objectives,  the  means  by  which  agencies  can  achieve  these  objectives,  and 
effective  execution  by  senior  agency  managers  are  critical  to  future  success  of  the  CCA 
(Cohen,  1994).  Today’s  CIOs  are  seen  as  business  executives  with  responsibilities  for 
harnessing  the  potential  of  IT  in  the  interest  of  their  organizations’  business  (Schafer, 
2001;  Scalet,  2000;  Periasamy  and  Seow,  1998;  Kom/Ferry,  1998).  CIOs  have  found  it 
necessary  to  address  key  management  factors  such  as  maintaining  a  holistic  business 
perspective  when  managing  IT,  being  knowledgeable  in  relevant  non-lT  disciplines, 
managing  people  effectively,  building  relationships  within  and  without  the  organization 
and  facilitating  communication  at  all  levels  (Periasamy  and  Seow,  1998). 

The  typical  CIO  in  the  private  sector  is  thought  of  as  providing  significant 
competitive  advantage  in  business  performance,  clearly  traceable  to  the  IT  strategy 
created,  sold  internally,  and  then  delivered  (Ulrich,  2001).  The  CIO  is  best  suited  to  lead 
the  enterprise  into  this  new  era;  more  than  other  top-level  executives,  he  understands  IT 
and  how  it  enables  business  strategy  (Ibid.).  With  appropriate  and  applicable 
performance  measures  in  the  federal  government,  such  CIO  achievements  are  possible. 
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but  will  depend  on  the  skills  and  abilities  of  the  CIO  and  the  organization  to  assess  risk, 
accept  it,  and  then  to  achieve  the  desired  results. 

Private  and  Public  Sector  Differences 

Differences  between  private  and  public  organizations  are  at  the  core  of  public 
administration  theory  and  have  been  the  topic  of  an  ongoing  stream  of  research.  Some 
public  administration  researchers  have  argued  that  a  dichotomy  such  as  public  and  private 
is  a  harmful  oversimplification  (Boseman,  1987,  Emmert  &  Crow,  1988).  However,  the 
sector  differences  presented  have  been  substantiated  empirically.  Differences  have  been 
found,  for  example,  in  personnel  management,  decision-making,  and  information  systems 
(Bretschneider,  1990). 

Sector  differences  have  been  linked  to  environmental  factors,  organization- 
environment  transactions,  and  internal  structures  and  processes  (Rainey,  Backoff,  and 
Levine,  1976).  The  public  sector  has  less  interaction  with  economic  markets,  and  this 
leads  to  behaviors  that  do  not  conform  to  the  incentives  and  punishments  associated  with 
market  controls.  There  are  more  constraints  on  procedures,  a  greater  tendency  toward 
formal  specifications  and  controls,  more  external  sources  of  formal  influence,  and  a 
greater  need  for  support  from  constituencies.  Many  of  the  activities  are  mandatory,  have 
a  broad  impact,  are  closely  scrutinized,  and  must  satisfy  unique  public  expectations. 
Public  sector  organizations  may  have  multiple,  and  oftentimes  conflicting  objectives;  less 
autonomy  and  control  over  decision-making  and  personnel;  greater  cautiousness;  more 
turnover  of  top  leaders;  more  difficulty  in  devising  incentives;  and  personnel  with  greater 
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variations  in  personality  traits  and  needs  (Bozeman  &  Bretschneider,  1986;  Rainy,  1983; 
Rainey  et  al.,  1976). 

The  position  of  the  CIO  has  been  explored  from  many  perspectives.  The  decision 
to  establish  a  CIO  in  each  agency  is  an  attempt  to  better  link  IT  to  agency  programs, 
while  providing  a  foundation  for  cross-agency  initiatives  (Hemon,  1996).  While 
determining  if  a  CIO  position  was  right  for  any  organization,  Hemon  observed  that 
research  and  analysis  should  be  integral  to  information  policy  formulation  and  review. 
Policy  makers,  “risk  confusion,  misunderstanding,  error,  and  redoing  work  if  they  do  not 
better  unite  the  study  of  government  information  policy  with  practice  (Hemon,  1996). 
Linking  private  sector  CIO  activity  and  public  sector  adoption  of  those  ideas  is  revealed 
in  a  March  2000  GAO  report: 

The  Clinger-Cohen  Act  required  major  departments  and  agencies  to  appoint  CIOs  and 
implement  IT  management  reforms  largely  grounded  in  successful  commercial  IT 
management  practices.  This  mirrors  the  evolution  of  the  CIO  position  in  industry 
where  it  has  largely  moved  from  solely  a  technical  support  focus  to  a  much  more 
executive  and  strategic  level  position  (GAO,  2000d). 

CIOs  are  common  in  many  private  and  public  sector  organizations,  although  the 
position  itself  is  not  the  solution  to  IT  problems  (Benchboard,  1996).  The  important  thing 
is  the  influence  that  CIOs  might  bring  to  bear  on  strategic  management  issues  by  ensuring 
that  IT  helps  resolve  existing  performance  problems,  and  identifying  opportunities  in 
which  IT  and  improved  information  management  can  enable  redesigned  work  processes 
and  service  delivery  methods  (Benchboard,  1996,  Kom/Ferry,  1998).  As  time  progresses 
and  organizations,  both  public  and  private,  put  into  practice  the  managing  methods  of  a 
CIO,  they  have  the  potential  to  benefit  from  the  experience. 


23 


CIO  Roles  and  Responsibilities 

Over  the  past  decade,  a  considerable  amount  of  attention  has  been  given  to 
defining  the  priorities,  roles,  and  responsibilities  of  the  CIO.  The  CCA  and  EO  13011 
imply  the  performance  of  private  sector  CIOs  could  prove  beneficial  to  the  federal 
government.  The  range  of  CIO  roles  and  responsibilities  extend  to  technical  and 
managerial  realms  (Periasamy  and  Seow,  1998). 

According  to  the  CCA,  PRA  '95,  and  0MB  Circular  A-130,  a  CIO  must  focus  on 
the  following  IRM  policy  issues; 

1)  Identifying  the  role  of  technology  in  strategic  plans, 

2)  Documenting  an  integrated  business  and  technology  architecture, 

3)  Determining  approaehes  to  IT  seeurity, 

4)  Creating  measures  to  balanee  release  of  information  with  demands  for  privacy, 

5)  Setting  IT  projeet  eost,  sehedule,  and  performance  goals, 

6)  Establishing  and  overseeing  a  eapital  investment  portfolio, 

7)  Setting  reeruitment/training  goals  in  new  technology  areas. 

As  this  list  shows,  the  CIO  must  address  many  major  policy  areas.  The  timing, 
interrelationships,  and  execution  of  this  body  of  agency  IRM  policy  should  be  understood 
in  order  to  better  view  the  requirements  plaeed  upon  the  CIO  and  their  functions 
(Beachboard,  1996).  Specifics  of  Federal  CIO  roles  and  responsibilities  are  included  as 
Appendix  B.  Additionally,  the  AF-CIO  offiee  reeently  updated  their  roles  and 
responsibilities  list.  For  referenee,  the  roles  are  listed  below  in  Table  2.  The  descriptions 
of  these  roles  can  be  found  at  Appendix  C. 
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Table  2:  MAJCOM  CIOs  and  HAF  Functional  CIO  Representatives 


Roles  and  Responsibilities,  2  Jan  2002 


1 

Capital  Planning  and  Investment  Control 

2 

Information  Technology  Acquisition 

3 

Performance  Measures: 

4 

Information  Assurance 

5 

Standards  &  Architecture 

6 

Strategic  Planning 

7 

Training  and  Education 

8 

Information  and  Knowledge  Management 

9 

Process  Improvement 

10 

Technology  Assessment 

11 

E-Govemment/E-Business 

The  role  of  the  CIO  is  to  be  accountable  for  planning  and  managing  all  of  an 
agency’s  information  management  resources  to  support  the  responsibilities  of  business 
managers.  In  this  context,  the  role  of  the  CIO  is  to  provide  executive-level  support  for 
the  agency’s  strategic  business  planning,  financial  planning  and  business  process  reform. 
From  the  literature,  it  appears  there  is  a  common  thread  of  CIO  skills  and  qualities  in 
private  sector  organizations  where  the  CIO  role  has  made  a  positive  impact  (Applegate, 
1992;  Yang.  1996;  Feeny,  1997;  GAO,  2000d;  GAO,  2001).  The  list  is  fairly  extensive 
so,  as  mentioned  in  the  intro  to  this  section,  the  list  is  limited  to  the  top  15,  in  no 
particular  order; 

1)  As  relationship  developer,  both  up  and  down  the  chain  of  authority. 

2)  As  role  model. 

3)  As  personnel  developer  and  staff  evaluator. 

4)  As  global  thinker,  big  picture  orientation,  long-range  strategic  visionary. 

5)  As  principled  organizational  leader. 

6)  As  technologist  and  standards  enforcer  /  initiator. 

7)  As  program  manager. 

8)  As  goal  setter  and  expectations  articulator. 

9)  As  financial  expert. 
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10)  As  demand  chain  expert. 

1 1)  As  change  agent. 

12)  As  reengineer. 

13)  As  researcher. 

14)  As  culture-carrier,  culture-champion,  culture-maker. 

15)  As  e-business  initiator  /  controller  a  “technology-opportunist”. 

The  specific  roles,  responsibilities,  and  authorities  assigned  to  CIOs  positions 
vary  reflecting  the  needs  and  priorities  of  the  organization  (Swanson,  2000).  This  is 
consistent  with  reports  presented  in  several  articles  reporting  their  experiences  with  IT 
management  and  the  CIO  roles  (Swanson,  2000;  Trimble  2000;  Rockhart,  1996; 

Grover,  1993).  The  evidence  suggests  there  is  no  one  right  way  to  establish  a  CIO 
position  and  that  leading  organizations  are  careful  to  ensure  that  information  management 
leadership  positions  are  appropriately  scoped,  defined,  filled,  and  implemented  to  meet 
their  unique  business  needs. 

Federal  Government  CIO  Certification 

Since  1997,  the  AF-CIO  office  has  been  experiencing  the  growing  pains  of 
creating  and  integrating  a  new  Executive  Level  position  into  the  hierarchy  (Trimble, 

1999,  2000).  This  is  a  possible  paradigm  shift  as  an  AF-ClO  is  now  responsible  to  treat 
the  utilizing  and  teaching  of  information  and  information  technology  as  a  resource  and 
mission  enhancer/enabler.  The  federal  government  has  taken  steps  to  certify  its  CIOs. 

The  Information  Resource  Management  College  (IRMC),  located  at  Fort  McNair 
in  Washington,  D.C.,  supports  a  CIO  Certificate  Program,  sponsored  by  the  DoD  CIO 
(ASD  31).  The  IRMC  has  been  designated  as  the  Department’s  flagship  for  information 
technology  management  training  for  senior  managers.  In  addition  to  the  two  primary 
programs  offered,  the  Advanced  Management  Program  and  the  DoD  CIO  Certificate 
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Program,  the  IRMC  has  established  the  Information  Security/Assurance  Certificate 
Program.  This  new  program  has  been  certified  by  the  National  Security 
Telecommunications  and  Information  Systems  Security  (NSTISS)  Committee  as  being 
compliant  with  the  Information  Systems  Security  Professionals  standard  (NSTISSI  No. 
4011).  The  IRM  College  is  one  of  only  four  schools  nationally  that  has  been  certified  as 
meeting  the  specified  NSTISSI  training  criteria  (Annual  Defense  Report,  2001). 

The  program  responds  to  the  requirements  set  forth  in  the  CCA  and  supports  an 
official  certification  to  recognize  individuals  receiving  advanced  education  in  the  Federal 
CIO  Core  Competencies.  The  CIO  Certificate  Program  is  organized  around  twelve 
subject  areas  directly  related  to  Federal  CIO  Core  Competencies  identified  by  the  Federal 
CIOC.  Table  3  below  contains  the  competencies  from  the  Federal  CIO  Council  Core 
Competencies  (2000).  Appendix  E  contains  the  complete  list  and  explanations  of  each  of 
the  Federal  CIO  Council  Core  Competencies  (2000). 


Table  3:  Federal  CIO  Core  Competencies  (2000) 


1 

Policy  and  Organizational 

2 

Leadership/Managerial 

3 

Process/Change  Management 

4 

Information  Resources  Strategy  and  Planning 

5 

IT  Performance  Assessment:  Models  and  Methods 

6 

Project/Program  Management 

7 

Capital  Planning  and  Investment  Assessment 

8 

Acquisition 

9 

E-Govemment/Electronic  Business/Electronic  Commerce 

10 

IT  security/information  assurance 

11 

Technical 

12 

Desk  Top  Technology  Tools 
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CIO  Challenges 

For  the  past  six  years,  the  Association  for  Federal  Information  Resources 
Management  (AFFIRM)  Emerging  Issues  Forum  has  conducted  annual  surveys  of  the 
senior  federal  information  technology  community  to  determine  the  most  critical 
challenges  facing  CIOs.  The  following  tables  (4  and  5)  represent  the  results  taken  from 
the  “The  Federal  CIO  -  Sixth  Annual  CIO  Challenges  Survey  (December  2001)”.  These 
AFFIRM  surveys  explore  how  the  top  challenges  facing  Federal  CIOs  today,  as  viewed 
by  senior  federal  government  IRM  officials  and  staff,  might  have  changed  from  year-to- 
year  as  well  as  changes  in  the  priority  of  the  top  ten  critical  technologies.  Approximately 
300  electronic  surveys  were  distributed  to  senior  information  technology  officials  and 
managers  at  federal  departments  and  agencies.  Each  of  these  two  tables  show  a 
comparison  between  2001  responses  and  the  prior  five  years.  In  a  few  cases,  specific 
challenge  statements  and  technologies  have  been  added  or  altered  to  reflect  current 
realities  (AFFIRM  2001). 
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Table  4:  CIO  Challenges  -  2001  Survey  Responses  and  Prior  Year  Comparisons 


2001 

Votes 

2001 

Ua  liking 

DESCRIPTION 

2000 

Ranking 

1999 

Ranking 

1998 

Ranking 

1997 

Ranking 

29 

1 

Using  IT  to  improve  service  to 
customers/stakeholders/citizens 

8 

5 

6 

7 

28 

2 

Making  the  business  and  cultural  changes 
necessarv  for  full  e-Government  transformation 

1 

1 

1 

1 

” 

"" 

27 

Hiring  and  retaining  skilled  professionals 

1 

1 

13 

26 

4 

Obtaining  adequate  funding  for  IT  programs  and 
projects 

4 

5 

"" 

25 

5 

Preventing  unauthorized  system  intrusions 
(hackers,  terrorists,  etc.) 

-y 

2 

— 

24 

6 

Formulating  or  implementing  an  agency  IT 
architecture 

6 

7 

■> 

1 

23 

7 

Building  effective  relationships  in  support  of  IT 
initiatives  with  agency  senior  executives  (agency 
head,  CFO,  etc.) 

7 

15 

9 

12 

19 

8 

Capturing,  organizing  and  making  accessible 
Agency  knowledge  and  expertise  (knowledge 
manasement) 

8 

10 

10 

18 

9 

Simplifying  business  processes  to  maximize  the 
benefit  of  technology  (see  note) 

10 

13 

10 

9 

17 

10 

Unifying  “islands  of  automation”  within  lines  of 
business 

1 

1 

" 

" 

" 

16 

11 

Aligning  IT  and  organizational  mission  goals 

12 

11 

5 

5 

15 

12 

Implementing  e-business/e-government  solutions 

2 

■y 

" 

" 

15 

13 

Providing  effective  IT  infrastructure  and  related 
services  (not  including  the  desktop) 

11 

9 

10 

6 

14 

14 

Implementing  IT  capital  planning  and 
investment  management  across  the  agency 

5 

5 

4 

2 

12 

15 

Assessing  and  developing  agency  IT  competence 
(training  and  education) 

9 

8 

9 

11 

12 

16 

Implementing  solutions  in  support  of 
Government  Elimination  Act  (GPEA) 

— 

— 

““ 

10 

17 

Measuring  and  reporting  past  performance 

15 

12 

9 

18 

Ensuring  public  access  to  information  vs.  the 
need  for  svstem  security 

13 

9 

8 

“ 

9 

19 

Controlling  IT  budgets 

17 

11 

7 

13 

8 

20 

Managing  or  replacing  legacy  systems 

11 

12 

9 

12 

8 

21 

Developing  agency- wide  IT  accountability 

18 

12 

13 

8 

22 

Identifying  and  reporting  specific  CIO/IRM 
measures/outcomes  under  the  Goyernment 

Performance  and  Results  Act 

16 

6 

6 

23 

Implementing  COTS  solutions  (ERP,  CRM,  etc.) 

19 

15 

- 

- 

24 

Planning  and  implementing  IT  disability  access 
solutions  into  existing  and  new  IT  systems 

20 

“ 

“ 

"" 

25 

Responding  to  outsourcing  (A76)  requirements 

- 

- 

- 

- 

Note:  replaced  “championing  BPR  as  a  precursor 
to  IT  decisions”  from  prior  suryeys 

Table  5:  CIO  Critical  Technologies  -  2001  Survey  Responses  and  Prior  Year  Comparisons 


2001 

Votes 

2001 

Ranking 

DESCRIPTION 

2000 

Ranking 

1999 

Ranking 

1998 

Ranking 

1997 

Ranking 

1996 

Ranking 

55 

1 

Security  Infrastructure 

1 

14 

1 

2 

2 

34 

2 

Internet  /  Intranet  /  Web  infrastructure 

2 

1 

2 

1 

1 

24 

3 

Knowledge  management 

3 

5 

3 

— 

— 

23 

4 

E-Mail 

14 

11 

13 

8 

10 

21 

5 

Internet/  Intranet/  Web  applications 

2 

1 

2 

1 

1 

20 

6 

Remote  and  mobile  computing  including 
personal  digital  assistants 

5 

4 

9 

sk 

sk 

19 

7 

Data  warehousing/data  mining 

6 

2 

4 

3 

4 

15 

8 

Security  Applications 

1 

14 

1 

2 

2 

14 

9 

Virtual  Private  Networks 

— 

— 

— 

— 

— 

12 

10 

Wireless  technology 

11 

11 

Records  management 

— 

— 

— 

— 

— 

11 

12 

Executiye  information  and  decision  support 
systems 

10 

6 

15 

10 

7 

10 

13 

Data,  yoice  and  video  convergence  (was  voice 
and  data  integration) 

4 

10 

12 

12 

12 

10 

14 

Storage  and  storage  networks 

— 

— 

__ 

__ 

— 

9 

15 

Video  solutions  (distance  learning,  virtual  office, 
desktop) 

13 

7 

— 

— 

— 

8 

16 

Workflow 

7 

5 

10 

6 

6 

8 

17 

Portal  technologies 

— 

— 

— 

— 

— 

7 

18 

Training  technology  and  applications 

7 

19 

COTS  applications  including  ERP,  CRM  and 
SCM  (was  COTS  deyelopment  S/W) 

14 

11 

11 

8 

1 

6 

20 

Middleware 

16 

9 

14 

11 

13 

5 

21 

Online  analytical  processing  (OLAP) 

19 

13 

14 

10 

14 

4 

22 

EC/EDI 

8 

3 

5 

5 

3 

■■ 

23 

IT  accommodation  -  disability  access  solutions 

11 

12 

“ 

“ 

“ 

3 

24 

Relational  databases 

16 

11 

14 

9 

8 

2 

25 

Next  generation  Internet 

9 

11 

8 

— 

— 

2 

26 

Voice  integration 

21 

— 

— 

— 

— 

2 

27 

Groupware 

21 

11 

8 

9 

8 

1 

28 

Application  Service  Provider  (ASP) 

12 

— 

— 

— 

— 

1 

29 

Imaging 

18 

10 

12 

7 

9 

0 

30 

LINUX 

19 

14 

- 

- 

- 

Private  Sector  CIOs 

The  purpose  of  this  section  is  to  highlight  the  many  challenges,  roles,  and 
responsibilities  of  the  position  of  CIO.  As  stated  earlier,  one  of  the  main  reasons  cited  in 
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the  Clinger-Cohen  Act  for  creating  the  CIO  position  was  the  notable  successes  of  the 
private  sector  CIOs  position. 

A  review  of  the  literature  regarding  private  sector  CIO  roles  and  responsibilities 
suggest  that  CIO  roles  are  extensive  and  continue  to  evolve;  this  implies  CIO  roles  are 
not  cookie-cutter  positions  where  a  “one  size  fits  all”  philosophy  works,  and  is  supported 
by  research  (Raghunathan,  et  al  1989;  Applegate  et  al,  1992;  Feeny  et  al,  1992;  Grover  et 
al,  1993;;  Earl  et  al,  1994;  Periasamy,  1998;  Swanson,  2000) .  Four  years  ago,  CIOs  said 
the  most  pressing  concern  they  faced  was  completing  information  technology  application 
projects  on  time  and  within  budget.  Priorities  have  shifted  and  CIOs  in  2000  say  their 
biggest  concern  is  improving  security.  John  J.  Davis  &  Associates  in  New  York 
surveyed  288  CIOs  to  determine  the  most  important  challenges  for  IT  departments 
(Computerworld,  2000).  The  results  of  their  study  are  summarized  in  Table  6  below. 
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Table  6:  Summary  of  Challenges 


Legend  for  Chart : 

A  -  Role 

B  -  2000 

C  -  1997 

A 

B 

C 

Improve  security  and 
integrity  of  systems/databases 

92% 

59% 

Complete  IT  application 
projects  on  time  and  on  budget 

88% 

82% 

Expand  communication 
bandwidth 

71% 

66% 

Set  systems  standards 
throughout  the  organization 

71% 

74% 

Increase  and  justify 

IT  investments 

50% 

65% 

Replace  aging  or 
incompatible  platforms 

42% 

68% 

(Computerworld,  2000) 


In  early  2000,  Infoworld  magazine  conducted  a  survey  of  77  CIOs  in  the  US, 
Europe,  and  Australia  to  find  out  how  CIOs  view  their  jobs.  The  results  are  in  Table  7. 


Table  7:  CIO  Duties 


Percent  of  CIOs  who  said  their  jobs 

include  these  duties: 

A 

B 

Technology  policy-maker 

94 .8% 

Functional  leader 

81.4% 

Systems  strategist 

68.5% 

Service  deliverer 

67.6% 

Change  leader 

64 .5% 

(Infoworld,  2000) 
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The  survey  results  from  practitioner-oriented  periodicals  appear  to  support 
academic  research  findings  that  CIO  duties  in  the  private  sector  continue  to  evolve  and 
change  as  the  IRM  areas  of  focus  change.  Federal  government  leadership  has  recognized 
the  research  findings  of  the  challenges  and  successes  experienced  by  private  sector  CIOs. 


IT  Investment,  Performance,  and  Productivity 

Extensive  research  has  been  conducted  in  the  effort  to  reach  a  globally  accepted 

method  in  which  to  assess  IT  investment  costs  to  organizational  productivity  payoffs  and 

performance  increases  (Cline  and  Guynes,  2001;  Keung,  et  al,  2001;  Bharadwaj,  2000; 

Brynjolfsson  and  Hitt,  2000;  Tallon,  et  al,  2000;  Sircar,  et  al,  2000;  Brynjolfsson  and 

Yang,  1999;  GAO,  1997b).  However,  in  public  practice,  performance  measures  are 

required  by  law  to  be  an  integral  part  of  any  federal  IT  program. 

Use  Capital  Planning  to  Improve  Performance:  Agencies  invest  more  than  $40  billion 
in  IT  to  support  some  26,000  information  systems.  Technology  now  affects  virtually 
every  aspect  of  the  way  the  Government  operates,  and  IT  investments  are  extremely 
important  to  the  success  of  e-gov  transforming  the  delivery  of  information  and 
services.  Agencies  will  use  capital  planning  and  investment  control  to  promote 
security  and  privacy  in  the  use  of  technology  and  guide  the  results  of  this  investment, 
and  ultimately  for  ensuring  results  from  other  capital  assets  as  well.  The  Government 
can  thus  achieve  outcomes  from  IT  investments  that  match  agency  strategic  priorities 
and  provide  real  benefits  for  the  American  people  (0MB,  2000). 

“The  need  to  achieve  high  returns  on  information  technology  (IT)  investments 
and  reduce  systems  development  risks  has  never  been  greater,  given  the  public’s  demand 
for  a  government  that  works  better  and  costs  less”  (GAO,  1995:  2).  Effectively  managing 
information  and  information  resources  in  today’s  dynamic  high-tech  environment  has 
been  a  formidable  challenge  for  the  federal  government.  As  the  rapid  advancement  of 
information  technology  has  progressed  through  time  it  has  revolutionized  the  way 
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businesses  and  governments  accomplish  goals  and  objectives.  The  fact  that  technology 
has  undergone  rapid  cycles  of  innovation,  causing  constant  change  in  the  federal  IT 
management,  compounds  the  difficulty  even  more.  Yet,  explosive  growth  and  use  of 
information  technology  has  quickly  outpaced  the  processes  and  federal  policies  that  serve 
to  guide  and  direct  its  utility,  “technology  and  its  applications  evolve  faster  than  policy¬ 
makers  can  develop  or  refine  policy  to  deal  with  ever-changing  and  ever-unfolding 
issues”  (Hemon,  1996:  2). 

Unreliable  Investment  Data 

IT  financial  data  are  unreliable  because  the  government  does  not  know  how 
much,  or  on  what,  it  spends  for  IT  (Cohen,  1994;  Johnson,  1997).  The  IT-related 
obligations  0MB  requires  agencies  to  report  systems  and  services  (0MB  Circular  A-1 1, 
1993),  totaled  approximately  $24.8  billion  in  fiscal  1998  (General  Accounting  Office, 
1999).  According  to  Bruce  McConnell,  director  of  OMB's  information  policy  branch,  the 
fiscal  1998  defense  IT  budget  was  $10.2  billion,  of  which  0MB  was  "unable  to  capture 
the  spending  on  embedded  systems"  (McConnell,  1997).  Fiscal  1998  IT  budgets  for 
DoD,  the  Defense  agencies,  and  the  services  are  summarized  in  the  Table  8  below 
(McConnell,  1997). 


Table  8:  1998  Federal  IT  Budget 


Fiscal  1998  IT  Budget 

DoD 

$10.2  billion 

DoD  Agencies 

$3.4  billion 

Air  Force 

$2.3  billion 

Navy 

$2.2  billion 

Army 

$2.3  billion 
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Obligations  also  are  difficult  to  quantify  because  IT  programs  are  funded  from 
two  sources:  procurement  accounts  and  operations  and  maintenance  accounts.  This 
information  is  neither  comprehensive  nor  collected  uniformly  government-wide  or 
agency  wide  (Johnson,  1997).  In  addition,  Johnson  observed  that  agencies  often  do  not 
separate  IT  obligations  from  total  program  dollars,  or  IT  obligations  are  lost  when 
lumped  into  administrative  accounts  (Johnson,  1997). 

For  instance,  the  legislative  and  judicial  branches  do  not  have  to  report  their  IT 
spending.  The  reporting  of  IT  obligations  under  $50  million  was  not  required  for 
embedded  weapons  systems  or  in  federally  funded  research  centers.  DoD  alone  estimates 
it  spends  between  $24  billion  and  $32  billion  annually  on  embedded  weapons  system 
software  that  is  also  not  reported  (GAO,  1997a).  If  they  were  known,  these  unaccounted- 
for  dollars  could  greatly  alter  the  government's  IT  investment  portfolio  (Johnson,  1997). 

Unknown  Return  on  Investment 

Return  on  investment  (ROI)  is  troublesome  as  well.  Agencies  did  not  usually 
quantify  accrued  IT  investment  benefits.  ROI  is  often  calculated  as  system  outputs  or 
activities  rather  than  in  improved  mission  performance  or  program  results  (for  example, 
33  percent  more  taxpayers  were  served  better,  faster,  more  conveniently,  or  situational 
awareness  and  accuracy  increased  50  percent)(Johnson,  1997). 

This  lack  of  quantitative  and  qualitative  understanding  about  IT  investment  led  to 
unsound  management  decisions  (Cohen  1994).  Poorly  managed  IT  investments  with 
inadequately  assessed  risks,  cost,  and  benefits  can  have  costly  consequences  (Cohen, 
1994) — and  even  impede  performance.  Conversely,  as  mentioned  above,  well-managed. 
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carefully  selected  IT  resources  that  focus  on  mission  needs  can  substantially  improve 
organizational  performance  while  reducing  cost. 

Performance  Measures 

The  CCA  requires  agencies  to  develop  and  employ  decision  criteria  for 
evaluating,  comparing,  and  prioritizing  IT  investments.  While  this  may  not  be  new  for 
some  government  processes  outside  of  IT,  Congress  has  mandated  it  specifically  for  this 
purpose.  This  perspective  is  approached  much  the  same  way  a  competitive,  profit¬ 
making  organization  would;  or,  as  Periasamy  believed,  “. .  .the  role  of  the  CIO  is  moving 
away  from  the  more  conventional  technical  and  managerial  position  to  a  business  and 
leadership  one”  (Periasamy  and  Seow,  1998).  The  new  business-like  perspectives  have 
included  eoneepts  sueh  as  return  on  investment,  eeonomie  analyses,  alignment  with 
mission  or  business  goals,  program  speeifie  measurements,  business  plan,  business  ease 
analysis  and  justifieation,  teehnieal  risks,  and  degree  of  proeess  redesign  and 
improvement  support. 

State  Government  CIOs 

The  researcher  recognizes  the  signifieanee  of  state  government  CIO  roles  being 
relevant  to  this  research  as  they  eontribute  to  the  establishment  of  the  CIO  position  and  to 
and  IRM.  While  state  government  CIOs  were  not  researched  to  lend  specific  support  to 
the  federal  CIO  role,  the  duties  they  perform  and  the  responsibilities  they  bear  are  similar 
to  that  of  federal  government  CIOs.  State  CIOs  have  established  a  single  organization 
that  focuses  on  and  represents  the  50  states’  IT  primary  interests  and  supports  CIOs 
endeavors  to  achieve  success  at  the  state  level. 
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The  National  Association  of  State  Chief  Information  Officers  (NASCIO) 


represents  state  CIOs  and  information  resource  executives  and  managers  from  the  50 
states,  six  U.  S.  territories,  and  the  District  of  Columbia.  State  members  are  senior 
officials  from  any  of  the  three  branches  of  state  government  who  have  executive-level 
and  statewide  responsibility  for  information  resource  management  (NASCIO,  2001).  The 
mission  of  the  association  is  to  shape  national  IT  policy  through  collaborative 
partnerships,  information  sharing  and  knowledge  transfer  across  jurisdictional  and 
functional  boundaries.  NASCIO  is  also  represented  at  the  Federal  CIO  Council.  The 
following  table  is  a  representative  list  of  overarching  focus  areas  that  NASIRE  is 
currently  promoting  for  the  50  states  (NASCIO,  2001): 

Table  9  NASCIO  Focus  Areas  (2001) 


NASCIO  Focus  Areas  (2001) 

Accessibility _ 

Communications  Infrastructure _ 

Digital  Government _ 

Information  Architecture _ 

Innovative  Use  of  Technology _ 

IT  Professional  Retention  &  Recruitment 

Public/Private  Partnership _ 

Service  Application _ 

State  Planning  &  Management  Initiatives 


As  governors  and  legislatures  seek  centralized  management  of  technology 
projects,  budget,  and  strategic  planning,  the  role  of  the  chief  information  officer  (CIO) 
has  become  an  increasingly  common  position  in  state  governments.  While  seeking  to 
build  a  consensus  among  elected  officials  and  heads  of  programmatic  agencies,  CIOs 
here  are  outlining  a  vision  of  improved  customer  service  through  IT  management  so  that 
state  governments  can  operate  more  efficiently,  thereby  benefiting  the  citizens  of  their 
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respective  states.  In  a  NASCIO  report  entitled  “The  Chief  Information  Officer” 
published  in  October  1998,  NASCIO  examined  the  scope  of  CIOs  responsibilities, 
strategies,  functions,  and  techniques  related  to  information  resource  management.  Of 
interest  for  this  section.  Table  10  below  provides  evidence  of  interest  to  this  study. 


Table  10:  State  Government  CIO  Roles  and  Responsibilities 


1 

Developing  economic  policy  using  IT 

2 

Managing  public  access  to  data 

3 

Statewide  process  re-engineering 

4 

Exercise  project  management  responsibilities 

5 

Authority  over  contracting  and  purchasing  functions  for  IT 

6 

General  legislative  advocacy  for  IT 

On  a  separate  but  related  report  involving  state  government  CIOs,  the  GAO 
prepared  an  Executive  Guide  to  assist  government  agencies  in  maximizing  success  of 
CIOs  (GAO,  2000).  This  Executive  Guide  centers  on  six  key  principles  from  the  study  of 
several  leading  organizations  (public  and  private)  that  have  been  successful  at 
implementing  IM  enterprises  processes.  The  principles  are  listed  in  the  following  figure. 
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SIX  KEY  PRINCIPLES  FOR  SUCCESS 

1.  The  ability  to  understand  and  recognize  information  management's  role  in 
creating  value.  In  other  words,  taking  steps  in  building  business  plans  that 
incorporate  information  management  and  appreciating  the  overall  influence  that 
it  has  on  strategic  direction. 

2.  Moving  to  visibly  define  the  role  of  the  CIO.  In  other  words,  clearly 
spelling  out  the  overall  duties  and  responsibilities  of  the  CIO  and  how,  exactly, 
he  or  she  fits  into  the  senior  management  team. 

3.  Finding  a  way  to  guarantee  CIO  credibility.  All  directives  from  the  CIO 
should  be  viewed  as  essential  to  the  organization.  CIOs  must  look  outside  of 
their  inner  circles  for  partnerships  and  peer  exchanges. 

4.  CIOs  must  balance  business  and  technical  needs  while  demonstrating 
results  -  and  successes.  They  must  build  a  mechanism  for  regular  feedback. 

5.  Information  management  must  be  able  to  adapt  quickly  to  the  ever 
changing  environment.  Structures  must  be  flexible  yet  still  designed  to  meet 
necessary  business  needs. 

6.  Building  information  management  talent.  Finding  ways  to  identify,  attract, 
train  and  keep  IT  talent. 


Figure  1:  GAO  Six  Principles  for  Success 

In  summary,  the  GAO  has  taken  notice  of  leading  organizations  that  have  taken  a 
proactive  role  in  establishing  information  as  a  resource  and  enabler.  They  continue  to 
create  the  technical  and  managerial  infrastructure,  strategic  outlook,  and  leadership  and 
business  culture  to  make  IT  initiatives  successful. 


The  Office  of  the  AF-CIO 

On  March  14,  1996,  the  DoD  Deputy  Secretary  designated  the  Assistant  Secretary 
of  Defense  for  Command,  Control,  Communications  and  Intelligence  (ASD(C31))  as 
DoD's  Chief  Information  Officer.  The  Chief  Information  Officer  of  the  Department  of 
Defense  (CIO,  DoD)  is  responsible  for  carrying  out  certain  provisions  of  the  ITMRA  on 
behalf  of  the  Secretary  of  Defense  (Office  of  the  Secretary  of  Defense,  1997).  Each 
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military  service  has  been  designated  as  an  executive  agency  by  the  ITMRA  and  under 
Presidential  Executive  Order  13011  is  required  to  designate  its  own  CIO  (CCA,  Section 
5123).  By  Order,  the  Secretary  of  the  Air  Force  appointed  the  Assistant  Secretary  for 
Acquisition  (ASAF(A))  as  the  Air  Force  CIO  (Secretary  of  the  Air  Force  Order  560.1, 
2001).  As  instructed  by  SAF  Order  560.1,  The  Principal  Deputy  Assistant  Secretary  for 
Business  and  Information  Management  (PDAS(BIM))  performed  the  AF-CIO 
responsibilities  on  a  full  time  basis  while  the  Deputy  Chief  of  Staff  for  Communications 
and  Information  (AF/SC)  acted  as  deputy  AF-ClO,  reporting  to  the  AF-CIO.  However, 
on  November  26,  2001,  The  Secretary  of  the  Air  Force,  Dr.  James  G.  Roche,  restructured 
the  AF-CIO  organization,  establishing  direct  reporting  between  the  Under  Secretary  of 
the  Air  Force  and  the  AF-CIO.  The  AF-CIO  is  the  principal  adviser  on  information 
management,  business  processes  and  information  technology  standards.  The  AF-CIO 
and  the  Deputy  CIO  are  responsible  for  overall  implementation  of  information 
technology  management  policy  for  the  USAF.  The  AF-CIO  organizational  structure  is 
included  below  in  Figure  2. 
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Major  IT 
Budgret  Issues 


SAF/OS 


AF-CIO 


SECAF  &  CSAF 


Mr.  John  Gilligan 


Air  Force  Council 


Deputy  AF-CIO 


Lt  Gen  John  L.  Wocmward  Jr, 


AF-CIO  Management  Board 


Air  Force'  Board 


HAF  2  -  Letters  and  MAJCOM  CIO  Rep 


HAF  Functional  Reps  and  MAJCOM  CIOs 


Air  Force  Group 


AF-CiOStaff|  AF/SC  I  MAJCOM  CIOs 


Figure  2:  AF-CIO  Organizational  Structure  (AF-CIO  Webpage,  28  Nov  01) 


The  AF-CIO  Management  Board  (CIOMB)  is  an  executive  forum  established  to 
improve  the  Air  Force  modernization,  use,  sharing,  and  performance  of  information 
resources  and  management  practices  through  re-engineering  and  capital  investment 
planning.  The  forum  oversees  matters  related  to  development  of  innovative  technologies, 
standards,  and  practices.  The  board  seeks  opportunities  for  improvement  in  multi¬ 
functional  cooperation,  common  infrastructures,  affinity  groups  for  technology  areas, 
coordination  of  information  needs  and  methodologies,  standards  and  guidelines  for 
information  management  and  technology.  The  Board  advises  the  AF-CIO  on  information 
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technology  investment  issues;  reviews  and  prioritizes  CIO  focus  areas;  and  provides  a 
forum  for  the  CIO  to  carry  out  his  role,  under  the  CCA. 

AF-CIO  Mission,  Vision,  and  Goals 

A  view  of  the  AF-ClO  Visions  and  Goals,  as  seen  from  the  AF-ClO  website,  is 
provided  in  Figure  3  below,  from  the  AF-ClO  website,  accessed  10  Nov  2001. 


Air  Force  CIO  Vision  and  Goals 

Mission 

-  Promote  the  most  effective  and  efficient  application,  acquisition  and  management  of 
information  technology  resources. 

Vision 

-  Enhanced  mission  performance  through  seamless  integrated  access  to  the  right  information 
anywhere,  anytime  —  One  Air  Force... One  Network. 

Goals 

-  Gain  the  greatest  benefits  of  information  technology  by  facilitating  business  process 
improvement  and  reengineering  efforts. 

-  Base  Air  Force  Information  Technology  (IT)  investment  decisions  on  sound  business  cases,  as 
well  as  approved  Air  Force  standards,  and  architectures. 

-  Ensure  the  availability  of  accurate,  trusted,  and  protected  information  to  the  right  person 
anytime,  anywhere. 

-  Integrate  CIO  IT  investment  reviews  and  recommendations  into  the  Air  Force  Planning, 
Programming,  and  Budgeting  processes. 

-  Ensure  Air  Force  personnel  possess  the  information  technology  skills  to  accomplish  their  Air 
Force  missions. 

Figure  3:  AF-CIO  Visions  and  Goals 

Air  Force  Information  Technology  Management  Plan 

"A  Government  that  works  better  and  costs  less  requires  efficient  and  effective 
information  systems."  (Executive  Order  13011,  1996).  This  line  not  only  begins  EO 
13011,  but  also  begins  the  USAE  Information  Technology  Management  Strategic  Plan 
(ITMSP)  by  previous  AE  CIO,  Art  Money.  The  Plan  begins  a  process  of  reengineering 
how  the  Air  Force  delivers  information  to  warfighters  and  those  who  support  them.  The 
Plan  is  also  intended  to  guide  the  development  of  detailed  plans  and  while  not  addressing 
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specific  programs,  initiatives,  or  budgets.  This  plan  meets  the  strategic  planning 
requirements  of  the  Clinger-Cohen  Act  (CCA),  Office  of  Management  and  Budget 
(0MB)  mandates,  and  Office  of  the  Secretary  of  Defense  directives.  Sections  1  and  11 
give  an  overview  and  outline  the  national  and  joint  influences  on  this  planning  effort. 
Section  111  describes  the  Air  Force  Chief  Information  Officer  (CIO)  organization  and 
processes. 

One  highlight  of  the  strategic  plan  directs  USAF  leadership  at  all  levels  to  focus 
on  four  effectiveness  characteristics  that  make  up  the  core  of  USAF  operations: 

1)  Make  Air  Force  missions  and  business  operations  better, 

2)  Build  communications  and  information  architectures, 

3)  Reengineer  our  processes  so  they  are  models  of  performance  and  efficiency, 

4)  Develop  and  nurture  strategic  partnerships  to  achieve  our  vision,  goals,  and 
objectives. 

These  broad  attributes  are  then  mapped  to  CIO  processes  and  strategies  that  are 
essential  to  successfully  accomplishing  the  USAF  core  competency  of  Information 
Superiority.  Measuring  effectiveness  is  outlined  in  Objective  1.4  of  the  ITMSP  and 
included  in  Figure  4  below. 


Objective  1.4-  Measure  information  technology  performance  in  mission  terms: 

Strategy  1.4.1  -  Implement  a  process  that  measures  information  technology  effectiveness  in 
mission  outcome  terms  (e.g.  sorties,  tonnage  moved). 

-  Measure  information  technology  effectiveness  on  a  continuous,  recurring  basis. 

-  Benchmark  performance  against  similar  activities  in  government  and  business. 

Strategy  1.4.2  -  Develop  models  and  analytical  tools  that  enable  the  prediction  of  mission 

improvements  from  information  technology  investment. 


Figure  4:  AF  ITMSP:  Objective  1.4 
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The  goal  of  these  strategies  clearly  places  effectiveness  performance  on  the 
shoulders  of  USAF  leadership.  Defining  and  developing  results-based  measurements 
will  inherently  be  required  by  personnel  at  the  “doer”  levels.  In  fulfilling  the 
responsibilities  of  the  CCA,  Paper  Reduction  Act  (PRA),  and  the  Government 
Performance  Results  Act  (GPRA),  the  Secretary  of  the  Air  Force  established  goals  for 
improving  the  efficiency  and  effectiveness  of  Air  Force  operations.  The  AF-CIO  advises 
the  Secretary  on  improving  the  effectiveness  of  Air  Force  operations  through  the 
effective  use  of  information  technology. 

The  AF-ClO  is  also  developing  a  guide  for  using  information  technology 
performance  measures  for  selecting  and  controlling  IT  investments.  The  goal  is  to  link  IT 
performance  measurement  to  the  requirements  and  budget  processes.  Essentially,  a  five 
percent  decrease  in  cost  coupled  with  a  five  percent  increase  in  efficiency  is  the 
expectation  of  Congress  (CCA,  1996).  In  addition,  a  rewrite  of  the  ITMSP  is  scheduled 
to  be  out  soon,  but  has  not  been  published  as  of  the  writing  of  this  thesis  study. 

AF-CIO  Focus  Areas 

The  AF-CIO  office  has  identified  nine  focus  areas  (see  Table  1 1  below).  The  AF- 
CIO  website  describes  the  USAF  Focus  Areas  that  are  listed  below,  with  detailed 
descriptions  found  in  Appendix:  D. 
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Table  11:  AF-CIO  Focus  Areas 


1 

Architecture 

B 

Business  Process  Reengineering 

B 

Capital  Planning  and  Investment  Control 

4 

Information  Resource  Management 

B 

Information  Technology  Acquisition 

6 

Performance  Measures 

B 

Standards 

8 

Strategic  Plan 

B 

Training  and  Education 

These  foeus  areas  aeeount  for  progressive  action  regarding  the  accomplishments 
of  the  AF-ClO  office  in  implementing  successful  IT  processes.  It  appears  the  USAF  is 
creating  a  logical  leadership  and  management  infrastructure  to  define  and  delegate  duties. 


Assessing  the  AF-CIO 

The  Federal  CIO  Position  Evaluation  Method  (FCPEM) 

The  FCPEM  is  an  evaluation  method  developed  by  Scott  Bernard  for  determining 
the  degree  to  which  a  federal  agency  has  complied  with  the  intent  of  the  CCA  in  creating 
their  CIO  position  (Bernard,  2001a).  The  assumption  is  that  if  agencies  are  in  technical 
compliance  with  the  CCA,  the  assessment  would  manifest  these  results.  It  is  further 
assumed  that  being  in  technical  compliance,  an  agency  can  therefore  aspire  to  substantial 
compliance  and  IT  success  (Beachboard,  1996).  To  support  the  development  of  this 
method,  the  ClO-related  mandates  in  Section  5125  of  the  CCA  were  identified  and 
interpreted  as  shown  in  Table  12  below  . 

The  FCPEM  operates  by  asking  evaluation  questions  that  determine  whether 
thirteen  CCA/CIO  mandated  roles  have  been  established  for  that  agency's  CIO  position. 
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Determining  the  degree  of  compliance  in  each  area  gives  both  a  functionally  specific,  and 
in  summary,  an  overall  indication  of  support  for  the  CIO  provisions  of  the  CCA.  Both 
compliance  with  and  variance  from  CCA-CIO  mandates  are  potentially  valuable 
information  for  policy  studies  on  what  the  effect  of  this  portion  of  the  law  has  been. 
Additionally,  the  identification  of  patterns  of  compliance  or  variance  within  the  twenty- 
three  agencies  listed  in  the  CCA  may  inform  discussions  regarding  future  federal  IRM 
law  and  guidance. 
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Table  12:  The  Federal  CIO  Position  Evaluation  Method 


CIO  Roles,  Per  the  CCA  The  Evaluation  Standard 
(Section  5125)  for  Each  CIO  Role 


Complexity  of 
the  CIO  Role 
Area 


Agency  establishes  a  CIO 

position/title. 

5125(a)(1)(A)&(B) 


CIO  designated  at 
Executive  Level-IV 
5125  (e) 


CIO  reports  directly 
to  the  agency  head 
5125  (a)(1)(A)&(B) 


IRM  is  the  CIOs 
principle  duty. 
5125(c)(1) 


CIO  ensures  efficient 
IRM  processes, 
including  reducing 
information  collection 
burdens  on  the  public 
5125  a(1)(c) 


CIO  supports  defining 
the  agency's  program 
information  needs, 
strategies,  systems, 
and  capabilities. 

5125  (a)(1)(c) 


CIO  heads  a  process  to 
evaluate  proposed 
agency  collections  of 
information. 

5125  (a)(2) 


CIO  provides  advice  to 
agency  head/management 
to  ensure  IT  is 
8  acquired  &  IRM  done 
I  AW  PRA  '95  and  agency 
head  priorities. 

5125  (b)(1)/51 22(a) 


CIO  develops, 
maintains,  facilitates 
9  an  integrated  agency 
IT  Architecture  (ITA) 
5125  (b)(2) 


CIO  monitors/evaluates 
IT  program  performance 
&  advises  continuation 
5125  (c)(2) 


CIO  participates  in  FY 
agency  strategic 
1 1  planning  &  performance 
evaluation  processes. 
5125  (c)(3) 


CIO  assesses  IRM  skill 
requirements,  develops 
strategies  to  rectify 
12  deficiencies,  w/  plans 
for  hiring,  training, 
professional  development 
5125  (c)(3)(A),(B)&(C) 


■ 

I 


Was  A  CIO  position 
formally  designated  and 
established? 


Is  the  CIO  a  member  of  the 
Senior  Executive  Service, 
Level  IV? 


Is  direct  ClO-agency 
head  reporting 
estabiished  in  writing? 


Does  the  designation 
document  make  IRM  the 
CIO's  principle  duty? 


Does  the  CIO  facilitate 
reviews  to  improve  IRM- 
related  processes, 
including  reducing  the 
public  information 
collection  burden? 


Is  there  a  CIO  &  CFO 
facilitated  process  for 
identifying  all  agency 
program  IT  needs, 
strategies,  systems, 
capabilities? 


Does  the  CIO  facilitate 
the  evaluation  of 
information  collections 
independent  of  CIO 
program  roles? 


Does  the  CIO  facilitate 
an  IT  Capital  Planning 
process,  advise  agency 
head/mgmt,  &  ensure  IT 
is  acquired  &  IRM/ITA 
are  done  lAW  PRA'95  & 
agency  head  priorities? 


Does  the  CIO  facilitate 
an  ITA  that  ties  to 
Capital  Planning  and 
follows  0MB  A-130/OMB 
97-16  format/guidance? 


Does  the  CIO  review  IT 
programs  for  <10% 
variance  in  cost, 
schedule,  performance? 


Is  there  an  agency  IT 
Strategic  Plan  and  is  it 
reflected  in  the  FY 
Strategic  Plan  and  the 
Performance  Report? 


Does  the  agency  have  a 
CIO-facilitated  IT 
Workforce  Plan  that 
addresses  needed  IRM 
skills,  training, 
hiring,  &  professional 
development? 


Does  the  CIO  report  in 
writing  to  the  agency  head 
each  year  on  how  I RM 
capability  is  improvino? 


OMB  96-02, 

EO-13011, 

PRA‘95 


Title  44 
U.S.  Code 
Section  5315 


PRA’95, 

OMB  Memo96-02 


OMB  Memo96-02 


PRA  '95, 
OMB  A-130, 
GPEA, 

GAO  Reports 


PRA  '95, 
OMB  A-130, 
GAO  Reports 


PRA  '95, 

OMB  A-130, 

OMBMemo96-02, 

FAR 


CIO  Council's  FEAF, 
OMB  Memo97-02, 
OMB  Memo97-16, 
OMB  A-130 


OMB  A-11, 
OMB  A-130, 
GPRA, 
PRA’95 


GPRA, 
OMB  A-11 


OMB  A-11, 
OMB  A-130, 
CIO  Council 


OMB  A-11, 
PRA’95 
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As  Bernard  describes,  the  FCPEM  is  not  meant  to  be  a  pass/fail  evaluation 
method.  It  is  meant  to  be  used  to  identify  whether  an  agency  is  complying  with  the  intent 
of  the  CCA  relative  to  how  the  agency  established  its  CIO  position.  Bernard  goes  on  to 
explain  that  with  the  assortment  of  organizational  forms,  cultures,  and  mission 
orientations  of  the  twenty-three  agencies  listed  in  the  CCA,  a  "cookie-cutter"  approach  to 
using  the  FCPEM  is  not  useful  or  appropriate.  Bernard  recommends  that  the  researcher 
using  the  FCPEM  as  an  analytic  tool  do  so  with  the  intention  of  documenting  an  agency's 
CIO/CCA  compliance  in  each  of  the  thirteen  areas  using  techniques  and  information 
appropriate  to  that  area.  “While  a  ‘comply/not  comply’  overall  rating  for  each  area  may 
be  appropriate,  amplifying  comments  in  areas  of  noncompliance  are  an  intended  part  of 
using  the  FCPEM”  (Bernard,  2001a). 

The  Federal  CIO  Position  Model  and  The  Parsons/Thompson  Model  of 

Organizations 

Bernard  also  developed  the  Federal  CIO  Position  Model,  which  serves  to  provide 
the  conceptual  relationship  of  the  CIO  position  when  illustrated  against  the 
Parsons/Thompson  Model  of  Organizations  (Bernard,  2001a).  The  Parsons/Thompson 
general  model  of  organizations  provides  an  aspect  of  structure  by  identifying  the  three 
distinct  levels  of  responsibility  and  control:  technical,  managerial,  and  institutional.  The 
CIO  Position  Model  builds  on  this  concept  by  relating  the  CIO's  general  roles  and 
responsibilities  (competencies)  to  the  organizational  level(s)  at  which  they  operate.  For 
this  research,  a  CIO  competency  area  was  defined  as  being  an  area  of  knowledge  that  is 
needed  to  successfully  perform  as  a  CIO  (Bernard,  2001a).  Together  they  provide  a 
theoretical  representation  of  the  organizational  context  and  the  policy  framework  that 
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makes  the  relationship  more  understandable.  The  model  further  involves  the  Federal  CIO 
Core  Competencies. 

The  Federal  CIO  Council  (CIOC)  updated  the  CIO  Core  Competencies  in 
September  2000.  This  comprehensive  list  is  included  as  Appendix  E  of  this  study.  The 
Federal  CIOC  has  included  in  its  CIOC  Fiscal  Year  2001-2002  Strategic  Plan,  section 
4.2,  to  update  the  Federal  CIO  Core  Competencies  every  two  years,  with  the  next  review 
to  take  place  by  September  30,  2002  (CIOC  Strategic  Plan,  2000). 

While  the  Federal  CIO  Core  Competencies  list  is  extensive,  the  researcher  did  not 
consider  this  list  to  be  fully  descriptive  of  the  Federal  CIO  position  because  it  lacked  an 
organizational  context.  Without  this  perspective,  the  roles  of  the  Federal  CIO  are  not 
seen  in  the  highly  dynamic,  multilevel,  complex  organizational  environment  that  is 
described  in  GAO  reports  (GAO,  2000b;  GAO  2000d)  and  as  identified  by  previous 
researchers  (Sweeny,  2000;  Periasamy  and  Seow,  1998;  Grover,  1993;  Kom/Ferry,  1998) 
In  selecting  a  model  of  organizations  to  be  used,  Bernard  considered  five  criteria: 

1 .  The  model  had  to  be  generic  enough  to  fit  the  variety  of  federal  departments, 
agencies,  and  commissions  that  comprise  the  executive  branch. 

2.  The  model  had  to  support  interaction  of  an  organization  with  its  environment, 
consistent  with  the  open-systems  orientation  of  this  study. 

3.  The  model  had  to  support  the  mapping  (cross-linking)  of  CIO  Core  Competencies 
to  organizational  levels. 

4.  The  model  had  to  recognize  that  business  processes  are  a  part  of  CIO  competency 
areas,  and  be  able  to  support  that  concept. 

5.  The  model  had  to  be  grounded  in  organizational  theory  to  support  use  in  scholarly 
research  ( Bernard  2001a). 

Bernard’s  perspective  of  the  organizational  model  of  organizations  that  were 
developed  by  Parsons  and  later  adapted  by  Thompson  was  selected  because  it  fit  these 
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criteria.  Case  study  validation  of  the  CIO  Position  Model  by  Bernard  further  supported 
this  choice. 

Bernard  fashioned  the  Parsons/Thompson  organizational  structure  dimension  to 
his  CIO  Position  Model  in  order  to  provide  a  conceptual  model  of  how  the  two  are  related 
and  to  reveal  that  a  basic  relationship  exists.  This  is  a  unique  approach  in  CIO  modeling, 
representing  organizational  theory  and  CIO  duties  or  competency  areas.  The  chief  reason 
for  placing  the  CIO  competency  areas  in  the  context  of  an  organizational  structure  is  to 
reveal  more  about  how  and  where  these  competency  areas  function  in  the  complex 
federal  agency  organization  (Bernard,  2001a). 

Bernard  validated  that  at  the  Technical  Level  of  the  CIO  Position  Model,  CIO 
functions  are  related  to  eore  business  proeesses,  information  protection,  and 
maintaining/enhancing  the  IT  inffastrueture.  Also  rational  decision-making  is  a  key  CIO 
activity  of  the  Teehnieal  Level.  At  the  Managerial  Level,  CIO  roles  involve  facilitation, 
resource  management,  and  the  maintenanee  of  interpersonal  relationships;  they  are  the 
focal  point.  Developing  and  maintaining  soeial  relationships  are  key  activities  of  the  CIO 
(Schafer,  2001).  Bernard  continues,  the  highest  of  the  CIO  Position  Model's  three 
organizational  levels,  the  institutional  level,  is  where  environmental  factors  predominate. 
Here  CIOs  are  engaged  in  both  rational  decision-making,  as  they  interpret  the  influence 
of  the  environment  on  core  business  processes,  and  socially-constructed  relationships 
with  external  actors. 

Bernard’s  CIO  Competencies,  in  Figure  5  below,  was  updated  to  reflect  the 
changes  made  to  the  Federal  CIO  Core  Competencies  list  in  October  of  2000  by  the 
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Federal  CIOC.  When  CIO  competency  areas  are  related  to  the  organizational  model's 
levels,  relationships  can  be  depicted  as  in  the  following  figure  as  example  relationships. 


Technical  Level.  "A  sub-organization  whose  problems  are  focused  around  effective 
performance  of  the  technical  function...  the  primary  exigencies  are  those  imposed  by 
the  nature  of  the  technical  task.  “ 

CIO  Competency:  Technical/Data  Management/Security/Architecture 
IT  Security/Information  Assurance 

Managerial  Level.  "Services  the  technical  sub-organization  by  mediating  with 
those  who  use  its  products,  and  procuring  the  resources  necessary  to  carry  out  its 
functions.  “ 

CIO  Competency:  Leadership/Managerial 

Project/Program  Management 
IT  Performance  Assessment:  Models  &  Methods 
Capital  Planning  and  Investment  Assessment 
Acquisition 

Institutional  Level.  "A  wider  social  system  which  is  the  source  of  the  meaning, 
legitimization,  or  higher-level  support  which  makes  the  implementation  of  the 
organization's  goals  possible.  “  (Thompson  ,  1967) 

CIO  Competency:  Policy 

Process  Change  Management 
Information  Resources  Strategy  &  Planning 
E-Government 


Figure  5:  (Revised)  Relating  CIO  Competencies  to  Organizational  Levels  (Bernard,  2001) 


This  relationship  model  might  be  sufficient  to  reflect  the  CIO  position  except  that 
it  must  be  eonsidered  that  CIO  competency  areas  may  extend  to  and  operate  in  more  than 
one  organizational  level.  Figure  6  below  provides  a  theoretical  updated  version  from 
Bernard’s  CIO  Position  Model  that  incorporates  the  current  version  CIO  Competency 
List  (September  2000)  and  the  Parsons/Thompson  tri-level  organizational  view,  informed 
by  key  actor  feedback  from  Bernard’s  research.  The  model  has  been  updated  with  the 
competencies  of  E-Govemment  and  IT  Security,  which  will  be  included  in  this  study.  It 
is  intended  to  be  descriptive  and  to  produce  a  visually  intuitive  depiction  of  the  CIO 
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position  that  indicates  that  the  12  roles  of  a  Federal  CIO  operate  across  different  levels  of 
a  generic  agency  organization. 


CT  CIO  Competency  Areas 


This  model  depicts  the  general  functioning  of  the  Federal  CIO 
Position  in  the  context  of  a  three-level  organization  structure 


Figure  6:  Federal  CIO  Position  Model  (Bernard,  2001) 


Some  of  the  CIO's  roles  relate  primarily  to  key  business  processes  that  are 
protected  in  the  core,  others  range  out  across  all  three  levels,  penetrating  the  core  and  also 
extending  out  into  the  institutional  environment  where  they  may  interact  with  external 
entities  (Bernard,  2001a).  Bernard  explains  some  CIO  roles  are  mostly  facilitation- 
oriented  and  therefore  exist  mainly  in  the  middle  management  layer.  The  implications  of 
the  role/level  relationships  and  interactions  between  CIO  roles  shown  in  the  CIO  Position 
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Model  are  not  explored  further  in  this  research,  but  are  discussed  further  with  respect  to 
areas  for  future  research,  in  Chapter  5.  With  this  CIO  Position  Model,  one  can  now 
visualize  the  multitude  of  IT-related  activities  that  a  CIO  must  attend  to.  An 
understanding  of  how  CIO  competency  areas  work  across  multiple  levels  of  the 
organization  also  creates  a  more  realistic  functional  context  for  viewing  the  specific 
mandates  of  the  CCA  for  Federal  CIO  oversight  and  process  facilitation  roles.  With 
these  tools,  the  researcher  can  now  proceed  to  use  the  FCPEM  method  to  evaluate  CIO 
positions  on  a  more  informed  basis. 

Process,  Outputs,  and  Outcomes 

This  research  is  also  concerned  with  exploring  and  elucidating  the  outcomes  of 
the  CCA  of  1996  on  the  AF-CIO  office,  and  ultimately  on  the  USAF.  This  section  is 
provided  in  order  to  present  the  processes  and  structure  for  contemplating  the  issues 
describing  the  outcomes  or  impacts  on  the  USAF. 

Process 

A  business  process  is  defined  by  Hammer  and  Champy  as  “a  collection  of 
activities  that  takes  one  or  more  kinds  of  input  and  creates  an  output  that  is  of  value  to  the 
customer”  (Hammer  and  Champy,  2001).  In  their  research  into  Business  Process 
Reengineering  (BPR)  Hammer  and  Champy  describe  that  the  concept  of  processes  gives 
most  managers  the  greatest  difficulty  because  most  people  focus  on  tasks,  jobs,  people, 
and  structures  instead  of  the  process  (Hammer  and  Champy,  2001). 

The  Department  of  Defense  (DoD)established  guidance  for  CIO  business 
processes  as  an  integral  part  of  a  CIOs  responsibilities.  The  DoD  Model  CIO  Study 
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(2000),  Section  6.0,  CIO  Processes,  is  described  below,  along  with  Table  13,  the 
suggested  CIO  processes: 


CIO  leadership,  roles  and  responsibilities  of  enterprise  IM  and  IT  programs  are 
directed  through  a  number  of  key  processes.  Given  that  effective  use  of  information 
and  technology  is  an  important  enabler  of  organization  success,  these  processes  are 
the  principal  methods  through  which  the  organization  implements  its  1/lT 
management  programs.  Assuring  that  the  processes  are  properly  conceived,  supported 
and  enforced  is  essential,  along  with  clear  links  to  CIO  roles  and  responsibilities 
(Department  of  Defense,  2000). 


Table  13:  Model  CIO  Study  2000,  Suggested  CIO  Processes 


1 

Strategic  Planning  for  Enterprise  Information  Management  &  Technology 

2 

Enterprise  Information  Architecture  Development  &  Implementation 

3 

Capital  Planning  &  Investment  Management 

4 

Enterprise  Management 

5 

Strategic  Sourcing 

6 

Training 

7 

Collaboration  &  Knowledge  Management 

8 

Technology  Strategy 

The  researcher  noted  these  suggested  processes  are  similar  to  previous  Table  2 
(MAJCOM  and  HAF  CIO  Roles),  Table  3  (Federal  CIO  Core  Competencies),  and  Table 
13  (AF-CIO  Focus  Areas).  The  following  Table  is  a  combined  version  of  those  listed 
previously,  in  order  to  view  their  similarities.  The  benefit  of  listing  the  tables  side-by- 
side  is  to  view  the  relationship  between  goals,  competencies,  focus  areas  and  roles  of 
CIOs  that  assist  in  describing  the  issues  in  order  to  present  the  structure  for  contemplating 
the  outcomes  or  impacts  on  the  USAF.  This  analytical  exercise  helped  to  frame  the 
survey  instrument  for  this  study. 
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Table  14:  Comparison  of  Tables 


Model  CIO  Study 
Suqqested  Processes 

MAJCOM  /  HAF 

CIO  Roles 

Federal  CIO  Core 
Comoetencies 

AF-CIO  Focus  Areas 

1 

Strategic  Planning  for 
Enterprise  Information 
Management  &  Technology 

Strategic  Planning 

Information  Resources  Strategy 
and  Planning 

Strategic  Plan 

2 

Enterprise  Information 
Architecture  Development  & 
Implementation 

Standards  &  Architecture 

Architecture 

Standards 

3 

Capital  Planning  & 

Investment  Management 

Capital  Planning  and 
Investment  Control 

Capital  Planning  and 

Investment  Assessment 

Capital  Planning  and 
Investment  Control 

4 

Strategic  Sourcing 

Information  Technology 
Acquisition 

Acquisition 

Information  Technology 
Acquisition 

5 

Enterprise  Management 

Project/Program  Management 

Information  Resource 
Management 

6 

Technology  Strategy 

Technology  Assessment 

ip™Mro>Miw«iiii!»lMayilnnil;^B 

7 

Training 
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Outputs 

Outputs  are  the  results  of  activities,  processes,  and  services  of  a  project.  They  can 
be  measured  numerically  or  in  terms  of  volume.  Typically,  outputs  are  the  number  of 
visits,  or  number  of  attendees  involved  in  a  project,  or  the  number  of  times  a  particular 
activity  has  been  conducted.  Outputs  measure  the  actual  work,  services  or  programs; 
actual  accomplishments  in  terms  of  delivery,  such  as  number  of  programs  offered, 
number  of  participants,  or  who  was  reached.  While  Hammer  and  Champy  explain 
outputs  as  being  the  valuable  item  for  the  customer,  this  research  endeavors  beyond 
outputs,  to  explore  the  outcomes  and  their  value  to  AF-CIO  office. 
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Outcomes 


Outcomes  are  the  results  or  impacts  of  the  activities,  processes,  and  services.  The 
outcomes  are  directly  related  to  the  research  problem,  where  the  researcher  is  usually 
most  interested  in  the  outcomes  that  are  most  reflective  of  the  problem.  Outcomes  are  the 
changes  or  results  due  to  the  execution  of  a  program  or  policy.  They  are  typically 
classified  in  three  areas;  1)  immediate  or  short  outcomes  are  produced  first,  2) 
intermediate  or  medium  outcomes  occur  later  as  a  result  of  immediate  outcomes,  and  3) 
long-term  outcomes  are  the  big  changes  the  program  ultimately  accomplishes.  This 
research  is  primarily  interested  in  exploring  USAF  outcomes  as  a  result  of  the  CCA,  at 
each  possible  level  that  may  be  described  by  the  survey  respondents. 

AF-CIO  Community  Survey  Results 

In  August  of  2001,  Dr,  William  B.  Rouse  reported  on  research  conducted  for  the 
AF-ClO  offiee.  Dr.  Rouse  is  the  H.  Milton  and  Carolyn  J.  Stewart  Chair  of  the  School  of 
Industrial  and  Systems  Engineering  at  the  Georgia  Institute  of  Technology.  Rouse 
conducted  interviews  with  members  of  the  AF-CIO  staff,  the  CIO  Management  Board 
(CIOMB)  and  the  CIO  Exeeutive  Committee  (EXCOM). 

The  purpose  of  the  study  was  two-fold:  1)  to  begin  a  dialogue  to  address  the  AF- 
CIO  office  greatest  concerns,  and  2)  to  measure  and  analyze  interviewee  answers 
regarding  the  role  of  the  AF-CIO  organization  (Rouse,  2001).  On  an  importance  scale  of 
0  to  10,  interviewees  were  asked  to  rate  the  AF-CIO  Roles.  The  results  of  the  interviews 
are  included  in  Figure  7  below. 
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Figure  7:  AF-CIO  Office  Survey  Results  (Dr  Rouse,  2001). 


In  agencies  in  which  staffs  readily  agree  on  a  common  vision,  IRM  discussions 
centered  on  how  to  apply  information  resources  in  the  most  effective  and  efficient 
manner  to  achieve  agreed  upon  mission  goals  (Holden,  1996).  Rouse’s  results  support 
Holden’s  findings  and  both  are  relevant  to  the  current  study.  They  support  AF-ClO 
office  direction  to  frame  CIO  roles  and  responsibilities  for  MAJCOM  and  Functional 
representatives  as  outlined  in  the  CIOMB  and  EXCOM  Charter,  Attachment  1.  The 
researcher  consulted  with  Rouse  about  these  results  and  their  relation  to  the  current 


research.  There  was  agreement  that  there  is  value  added  in  conducting  the  current 
research  to  not  only  look  at  roles,  but  also  research  their  impacts  on  the  USAF  from  the 
perspective  of  the  MAJCOM,  Functional,  and  base-level  leadership.  Also,  the  current 
research  adds  the  context  of  organizational  management  theory  to  assist  in  formulating 
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and  explaining  the  results.  Both  research  projects  have  the  potential  to  contribute  to  and 
influence  the  behavior  of  USAF  leaders  and  future  decisions  made  by  the  AF-CIO  office. 

Organizational  Culture  and  Change 

The  federal  government  experienced  several  IRM  policies  changes  in  the  1990’s. 
Viewing  how  the  CCA  impacted  the  USAF  is  important  to  this  study.  Organizational 
culture  has  been  found  to  play  an  important  part  in  the  outcome  of  federally  mandated 
IRM  policy  in  federal  agencies  (Benchboard,  1996;  Holden,  1996).  Benchboard 
discovered  that  agency  culture  is  critical  to  achieving  IT  success  and  that  culture  largely 
determines  the  relationship  between  the  IT  organization  and  the  program  areas  and 
policies  they  are  to  support  (Ibid.).  Holden  found  that  organizational  culture  played  a 
surprisingly  strong  role  in  enforeing  ageney  IT  management  policy. 

Dr.  Edgar  H.  Schein  defined  modem  organizational  culture  theory  saying  that, 
“eulture  should  be  reserved  for  the  deeper  level  of  basic  assumptions  and  beliefs  that  are 
shared  by  members  of  an  organization,  that  operate  eonsciously,  and  that  define  an 
organizations  views  of  itself’  (Shatritz  and  Ott,  1996:  432).  Organizational  culture 
theory  describes  what  occurs  in  organizations  and  provides  possible  applications  for  the 
leadership  of  the  organization.  In  faet,  this  understanding  of  the  theory  is  so  important 
that  Schein  states  that  the  only  thing  of  real  importance  that  leaders  do  is  create  and 
manage  culture  (Ibid).  He  feels  that  the  process  of  culture  creation  and  management  are 
the  essence  of  leadership.  If  leaders  want  to  start  evolutionary  change  processes,  they 
must  be  adaptive. 
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Culture  is  also  described  by  Gibson  as  “  the  property  of  an  independently  defined 
stable  social  unit,  and  is  a  learned  product  of  group  experience  to  be  found  only  where 
there  is  a  definable  group  with  significant  history”  (Gibson,  1997:  30).  Organizational 
culture  differentiates  the  organization,  provides  a  sense  of  identity  to  members,  facilitates 
commitment,  enhances  social  system  stability,  and  promotes  sense-making  and  control 
for  attitudes  and  behaviors  (Schafritz  and  Ott,  1996:  433). 

The  concept  of  organizational  culture  theory  extends  to  measures  of 
organizational  effectiveness,  determinants  of  structure  and  design,  power  and  politics, 
intergroup  conflicts  and  conflict  resolution,  and  organizational  development  and  change 
(Schafritz  and  Ott,  1996).  The  concept  of  culture  is  particularly  important  when 
attempting  to  manage  organization-wide  change.  Practitioners  have  seen  that  despite  the 
best-laid  plans,  organizational  change  must  include  not  only  changing  structures  and 
processes,  but  also  changing  the  corporate  culture  as  well  (Hammer  and  Champy,  2001). 
The  difficulty  in  creating  a  culture  is  made  even  more  complex  when  attempting  to  bring 
about  a  significant  cultural  change  (Gibson  et  al,  1997:  35).  Organizational  culture 
theory  suggests  an  effective  way  of  bringing  about  a  change  in  people’s  values  and 
beliefs  is  to  focus  on  changing  behavior;  however,  a  change  in  behavior  does  not 
necessarily  bring  about  a  change  in  culture. 

Research  Questions 

Having  reviewed  the  literature  relevant  to  the  purposes  of  this  thesis  research,  five 
major  questions  were  formulated  regarding  the  outcomes  of  the  Clinger-Cohen  Act  and 
the  AF-CIO  office  and  their  impact  on  the  USAF: 
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1)  Are  the  USAF  implementation  efforts  consistent  with  the  intent  of  the  CCA? 

2)  To  what  extent  can  a  model  of  the  AF-CIO  position  illustrate  the  level  of  the 
AF-CIO  office  implementation  efforts? 

3)  What  outcomes  can  be  found  as  a  result  of  the  CCA  and  the  AF-CIO  office 
requisite  to; 

A)  The  Federal  CIO  Core  Competencies, 

B)  The  performance  based  aspects  of  the  CCA,  Section  5123, 

C)  Key  IRM  achievement  areas  from  the  public  and  the  private  sector, 

D)  The  roles  and  responsibilities  of  the  AF-CIO 

4)  Have  changes  in  USAF  operations  occurred  as  a  result  of  instituting  an  AF-CIO 
office? 

5)  How  have  these  changes  impacted  the  USAF? 

Summary 

The  goal  of  this  literature  review  section  is  to  describe  what  has  been  learned 
about  this  topic  and  to  provide  the  framework  for  the  research  methodology.  This 
chapter  reports  what  has  been  published  about  the  Clinger-Cohen  Act  (CCA),  CIOs,  and 
the  management  of  information.  This  section  summarizes  a  high-level  view  of  CIO  Core 
Competencies,  or  focus  areas,  of  government  CIOs,  private  sector  CIOs,  and  the  AF-CIO. 
Further,  the  relationship  between  the  roles  and  responsibilities  of  CIOs  are  associated 
with  an  organizational  context  and  policy  framework  that  makes  the  relationship  more 
understandable. 

To  correct  the  shortcomings  associated  with  current  management  of  IT 
investments.  Congress,  over  the  past  decade,  enacted  several  pieces  of  legislation 
requiring  federal  agencies  to  implement  IT  management  and  performance  measures  in 
their  business  processes  to  ensure  the  proper  oversight  and  management  of  IT 
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investments.  The  Clinger-Cohen  Act  (CCA),  which  created  the  office  of  the  Federal 
CIO,  is  the  object  of  this  study. 

An  examination  of  the  field  revealed  a  large  body  of  prescriptive  research 
regarding  the  roles  and  responsibilities  of  the  CIO  with  very  few  research  studies  on  the 
impacts  of  the  CCA  on  federal  agencies.  However,  IRM  and  Federal  Policy  researchers 
such  as  Bernard,  Beachboard,  Hemon,  and  Holden  have  reported  impacts  of  federal  IRM 
legislation  and  policy  on  the  federal  government.  This  study  refers  to  their  findings  as  a 
basis  for  exploring  the  impacts  of  the  CCA  on  the  USAF. 

The  IT  management  provisions  in  the  CCA  were  intended  to  improve  government 
financial  management  IT  initiatives  (Beachboard,  1996).  Built  on  practices  which  are 
common  to  leading  publie  and  private  organizations  (GAO,  2000b;  GAO,  2000d;  United 
States  Congress,  1996),  the  CCA  is  designed  to  assist  in  foeusing  senior  management 
attention  on  seleeting  well  designed  projeets  with  sound  business  justifieations  while 
mitigating  risks  as  IT  investments  as  they  proeeed  through  development,  and  evaluating 
aetual  performanee  improvement  results  attributable  to  the  investments. 

To  provide  an  understanding  of  the  CCA's  legislative  intent  with  respect  to  the 
federal  CIO  position,  the  CCA  was  reviewed  along  with  other  federal  IRM  policy  and 
other  related  guidanee.  Information  relating  the  CCA  and  Executive  Order  13011,  which 
created  the  CIO  Couneil  is  also  presented.  The  federal  policies  are  linked  to  the  AF-CIO 
focus  areas  and  are  examined  along  with  experienees  of  federal  and  state  government 
agencies,  and  private  sector  organizations. 

Beyond  technical  adherence  of  the  CCA,  how  can  substantive  compliance  be 
assessed  and  the  impacts  on  the  USAF  be  explored?  The  researcher  recognizes  that 
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policy  compliance  alone  does  not  provide  a  full  assessment  of  the  AF-CIO  office  and  it’s 
affects  on  USAF  operations.  Flowever,  it  does  provide  a  basis  for  further  exploring  the 
impact  of  the  CCA  on  the  USAF. 

With  the  key  policy  directives  identified,  IT  performance  factors  elucidated,  CIO 
roles  and  responsibilities  explained,  and  the  conceptual  model  illustrated,  the  researcher 
can  now  establish  a  methodology  in  which  to  address  and  answer  the  research  questions. 
The  methodology  can  be  used  to  gather  data  and  explain  the  impact  of  the  CCA  on  the 
USAF. 
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III.  Methodology 


Overview 

The  methodology  chapter  describes  how  the  research  for  this  thesis  is  structured 
and  performed.  This  social  research  study  is  inductive  and  qualitative  in  nature;  it  seeks 
an  ethnographic  approach  to  developing  an  understanding  of  a  topic  rather  than  testing  a 
theory.  Qualitative  research  deals  with  opinion  statements  leading  to  generalizations. 

The  nature  of  the  data  was  largely  qualitative  due  to  this  being  a  grounded  theory  study. 
However  tests  of  proportions  between  populations  are  also  conducted. 

According  to  Babbie,  social  research  serves  many  purposes.  Three  of  the  most 
common  are  exploration,  description,  and  explanation  (Babbie,  1998:90).  This  research 
is  an  exploration  of  the  Clinger-Cohen  Act  (CCA)  and  its  impact  on  the  USAF. 
Exploratory  studies  are  most  typically  done  for  three  purposes:  (1)  to  satisfy  the 
researcher’s  curiosity  and  desire  for  better  understanding,  (2)  to  test  the  feasibility  of 
undertaking  a  more  extensive  study,  and  (3)  to  develop  the  methods  to  be  employed  in 
any  subsequent  study  (Babbie,  1998:  90).  This  study  addresses  all  three. 

This  study  is  exploratory  because  the  method  of  complying  with  many  of  CCA's 
mandates  continues  to  be  interpreted  by  Congress,  the  presidential  administration,  and 
other  agencies.  Examples  of  this  are  the  December  2000  revision  to  0MB  Circular  A- 
130  that  now  incorporates  CCA  mandates,  and  congressional  inquiry  in  2000  into 
CCA/CIO  compliance.  In  particular,  an  October  2000  investigative  report  by  Senator 
Fred  Thompson  (R-Tennessee)  found  that  "Roles,  reporting  relationships,  and  boundaries 
of  authority  among  CIOs  within  large  executive  agencies  and  departments  are  not  clearly 
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established"  (U.S.  Senate,  2000).  Also,  the  January  2001  revision  of  SAF  Letter  Number 
560.1,  The  Chief  Information  Officer  of  the  Air  Force,  indicates  further  refining  of  the 
AF-CIO  office  continues  to  be  necessary.  As  mentioned  earlier,  on  November  26,  2001, 
The  Secretary  of  the  Air  Force,  Dr.  James  G.  Roche,  restructured  the  AF-ClO 
organization,  establishing  direct  reporting  between  the  Under  Secretary  of  the  Air  Force 
and  the  AF-CIO.  Finally,  a  recent  amendment  to  the  CCA,  Section  331,  “Additional 
Information  Technology  Responsibilities  of  Chief  Information  Officers”,  as  well  as 
published  developments  on  the  AF  ITMSP  (which  have  not  yet  been  released)  are 
representative  of  the  field  of  study  which  continues  to  evolve. 

Research  Approach 

To  gain  greater  insight  into  a  particular  problem,  it  is  often  advantageous  to 
question  knowledgeable  individuals  about  it.  This  exploratory  study  is  conducted  in-part 
using  a  narrative,  web-based,  open-ended  questionnaire.  A  content  analysis  of  the 
responses  to  this  non-reactive,  open-ended  questionnaire  approach  is  meant  to  help  guide 
the  exploration,  clarify  concepts,  and  build  theory  from  the  evidence.  An  advantage  of  a 
self-administered  survey  is  the  prospective  anonymity  of  the  respondent  which  could  lead 
to  greater  validity  overall  (Babbie,  1998:257).  Since  the  questionnaire  is  web-based, 
there  is  no  interviewer;  interviewer  bias  is  eliminated  at  the  time  of  recording. 

This  qualitative  study  also  uses  archival  data  in  the  form  of  federal  government 
and  USAF  directives  that  both  govern  the  AF-CIO  office  and  management  staff  in  their 
roles  and  responsibilities,  as  well  as  explore  the  issues  related  to  the  CCA  and  its  impact 
on  the  USAF.  Reports  on  the  impacts  of  federal  IRM  policy  on  federal  government 
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agencies  were  reviewed.  Finally,  interviews  were  conducted  with  notable  researchers 
who  have  influenced  the  field  of  federal  IRM  policy  and  management. 

Exploratory  research  is  theory  building  rather  than  theory  testing  (Dooley,  2001; 
253).  As  one  researcher  noted,  “Model  building  is  an  ongoing  process.  Because  a 
participant  observer  does  not  go  into  the  field  with  a  hypothesis,  the  end  point  of  such  a 
study  is  not  always  obvious.  The  construction  of  the  model  signals  the  end  of  the  study, 
and  the  first  attempts  at  model  building  usually  are  made  long  before  the  researcher 
leaves  the  field”  (Browne,  1976,  81). 

As  Dooley  mentions,  “qualitative  studies,  with  the  least  control,  risk  all  of  the 
internal  validity  threats”  (Dooley,  2001 :  269).  This  could  make  for  rather  difficult 
research  design  construction;  however,  relying  on  previous  research,  controlling  for 
internal  validity  through  instrument  manipulation,  choosing  the  correct  sample  frame  and 
sample  number,  and  selecting  a  qualitative,  exploratory  approach  institutes  what  Dooley 
describes  as  “clever  and  persistent  puzzle  solving”  (Dooley,  2001:  278). 

Focusing  on  the  impacts  of  the  CCA  on  the  USAF,  the  researcher  selected  an 
open-ended  questionnaire;  the  respondent  uses  his/her  own  words  to  answer  most  of  the 
questions.  This  research  examines,  through  content  analysis  and  statistical  analysis,  the 
respondent’s  answers  and  takes  into  account  the  environmental  and  organizational 
aspects  of  the  impacts  of  the  CCA  on  the  USAF. 

The  researcher  is  interested  in  learning  about  the  perceptions  and  experiences  of 
Executive  and  Managerial  Level  information  professionals  in  various  echelons  in  the 
USAF,  and  having  a  direct  relationship  to  the  CCA  and  the  AF-CIO  office.  The  data 
collected  from  the  respondents  was  interpreted  by  a  number  of  IRM  graduate  students  at 
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the  Air  Force  Institute  of  Technology  (AFIT)  specializing  in  CIO  concepts,  to  determine 
appropriate  classifications  in  determining  technical  and  substantive  compliance  with  the 
CCA,  building  on  the  evolutionary  model  for  the  AF-CIO  position,  and  exploring  the 
resultant  outcomes  to  the  USAF. 

This  research  also  employs  a  web-based  open-ended  questionnaire  survey 
method.  There  are  many  advantages  of  a  survey  in  information  systems  research 
(Benbasat,  1987).  One  advantage  is  an  appropriate  way  to  research  an  area  in  which  few 
previous  studies  have  been  carried  out.  As  revealed  in  the  literature  review,  this 
population  has  not  been  studied  relative  to  the  CCA.  Little  research  has  been  conducted 
about  the  impact  of  the  CCA  on  the  office  of  the  AF-CIO  and  it’s  influence  on  the  USAF. 
For  such  unexplored  subjects,  the  questionnaire  method  helps  to  discover  new  concepts 
and  relationships  between  them  for  further  research. 

Population 

The  research  methodology  uses  a  web-based,  self-administered  survey.  E-mail 
notifications  for  selection  to  participate  in  the  survey  were  sent  to  senior  infomiation 
executives  and  members  of  top  management  within  the  USAF.  The  population  of  interest 
includes  members  of  the  AF-CIO  staff,  members  of  the  AF/SC  staff,  members  of  the  AF- 
CIO  Executive  Committee  (EXCOM),  and  base-level  individuals  who  are  responsible  for 
CIO  related  duties,  such  as  USAF  Communications  Squadron  Commanders  and  Group 
Commanders. 

The  population  was  separated  into  2  groups  -  Executive  Level  and  Managerial 
Level.  The  Executive  Level  group  includes  AF-CIO  staff  members,  AF/SC  staff 
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members,  and  EXCOM  members,  including  all  USAF  Major  Command  (MAJCOM) 
CIOs  and  Headquarters  Air  Force  (HAF)  Functional  CIO  representatives.  The 
Managerial  Level  group  includes  base-level  individuals  who  are  responsible  for  CIO 
related  duties,  such  as  USAF  Communications  Squadron  Commanders  and  Group 
Commanders. 

The  research  establishes  that  the  population  consists  of  2  mutually  exclusive  and 
independent  groups  and  that  the  independent  group  responses  can  be  expressed  as  a 
single  population.  In  other  words,  the  groups  are  equal  in  agreement  as  to  their  roles  and 
functions,  and  that  they  are  united  in  purpose.  This  does  not  serve  to  specifically  answer 
any  of  the  research  questions.  However,  it  aseertains  that  in  the  organizational  structure 
of  the  AF-CIO,  the  Exeeutive  Level  establishes  the  managerial  roles  and  the  institutional 
goals  and  the  Managerial  Level  is  then  expeeted  to  implement  the  Executive  Level 
direetions. 

Exeeutive  Level  eontaet  was  obtained  through  the  AF-ClO  website.  A  list  of 
potential  base-level  respondents  was  produeed  by  gathering  organizational  information  of 
USAF  members  with  the  C33S4  Air  Force  Specialty  Code  (AFSC).  USAF 
commissioned  offieers  with  the  C33S4  AFSC  are  designated  Commanders  in  the 
Communieations  and  Information  career  field.  The  Managerial  Level  pool  of  potential 
base-level  respondents  was  obtained  from  an  Air  Force  personnel  system  database 
administered  by  staff  from  the  Air  Force  Institute  Technology  Registrars  office. 
(AFIT/RR).  The  list  was  provided  to  the  researeher  once  the  Air  Force  Personnel  Center 
(AFPC/DPSAS)  at  Randolph  AFB,  TX,  approved  the  survey  request. 
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For  this  research,  1  polled  the  entire  population.  Surveying  the  entire  population 
facilitates  exploration  across  a  wide  variety  of  organizations  and  reduces  the  effects  of 
deviant  or  disconfirming  cases  from  different  respondents  (Babbie,  1998;  462).  The 
population  size  is  estimated  to  be  approximately  250  individuals.  Respondents  were 
notified  via  electronic  mail  that  they  had  been  selected  to  participate  in  the  survey  (see 
Appendix  G).  It  was  left  to  the  e-mail  recipient’s  discretion  whether  or  not  to  participate 
in  the  survey  research. 

Data  Collection  Method 

In  choosing  a  data  collection  method  it  is  important  to  look  at  both  the  objectives 
as  well  as  the  target  population.  Harmonizing  the  research  objectives  and  the  target 
population  will  allow  for  selecting  the  best  possible  survey  method  that  will  work  better 
other  methods  (Babbie,  1998:89).  Since  Air  Force  IT  and  IRM  executives  are  located 
worldwide,  and  are  frequently  away  from  their  work  areas  on  temporary  duty,  the  web- 
based  survey  was  judged  to  have  the  best  likelihood  for  success.  This  is  supported  by  the 
thesis  research  conducted  by  Franke  in  the  area  of  web-based  survey  instruments  (Franke, 
2001:73). 

The  goal  of  a  survey  is  to  gain  specific  information  about  either  a  specific  group 
or  a  representative  sample  of  a  particular  group.  Results  are  typically  used  to  understand 
the  attitudes,  beliefs,  or  knowledge  of  a  particular  group.  Also,  “A  review  of  all  possible 
methods  revealed  that  the  questionnaire  is  more  advantageous  than  other  means  of  survey 
data  collection”  (Biros  and  Cole,  1992:  27).  This  observation  was  based  on  consideration 
of  cost,  opportunity,  and  anonymity,  as  well  as  the  time  afforded  the  respondents. 
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Furthermore,  Franke  concluded; 


An  overall  analysis  of  the  findings  of  this  study  makes  it  reasonable  to  conclude  that 
paper-based  and  computer-based  surveys  can  be  considered  equivalent  in  a  voluntary 
self-report  environment.  Additionally,  evidence  shows  that  complexity  and  format 
differences  between  computer-based  surveys  do  not  significantly  affect  responses  or 
response  rates.  Finally,  the  data  suggests  that  it  is  improbable  that  significant  bias  was 
introduced  into  survey  results  based  on  survey  method  of  administration,  gender,  or 
military  commission  (Franke,  2001). 

Based  on  pre-testing,  it  is  expected  that  the  web-based  survey  would  take 
approximately  30  minutes  to  complete.  To  protect  respondent  anonymity,  responses 
were  input  directly  in  a  database  with  limited  capability  to  identify  groups,  and  no 
capability  to  identify  individuals.  The  survey  is  hosted  on  a  network  server  operated  and 
maintained  by  the  Air  Force  Institute  of  Technology  with  no  requirement  for  external 
support.  The  data  collected  from  the  respondents  are  interpreted  by  a  number  of  IRM 
graduate  students  enrolled  at  the  Air  Force  Institute  of  Technology  (G1R-02M)  to 
determine  appropriate  classifications  in  creating  an  evolutionary  model  on  AF-ClOs  and 
their  outcomes  on  the  USAF. 


Questionnaire  Design 

A  questionnaire  was  constructed  to  address  all  five  research  questions. 
Screenshots  of  the  questionnaire  are  located  in  Appendix  F.  The  web-based  survey  is 
designed  in  three  segmented  parts  that  address  each  of  the  questions. 

The  Part  1  questions  are  designed  to  answer  Research  Questions  #3,  #4,  and  #5. 
Part  1  of  the  survey  (outcomes)  includes  open-ended  questions  and  requires  some 
endurance  and  patience,  as  well  as  a  critical  and  coherent  thought  process  in  order  to 
accomplish.  Open-ended  questions  in  Part  1  posed  the  question  in  the  form  “what 


69 


changes  have  occurred  as  a  result  of  the  CCA. . in  specific  areas  of  interest  to  the 
researcher.  Part  1  open-ended  questions  were  designed  to  elicit  and  explore  the  outcomes 
of  the  AF-CIO  office  on  the  USAF,  based  on  the  Competencies  created  by  the  Federal 
CIO  Council,  the  performance  requirements  of  the  CCA  (section  5123),  and  resulting 
outcomes  reported  from  public  and  private  sector  research  and  reports. 

The  Part  2  questions  are  designed  to  answer  Research  Question  #2.  Continuing 
Bernard’s  research.  Part  2  of  the  questionnaire  builds  on  the  conceptual  model  of  the 
Federal  CIO  Position  Model.  This  research  demonstrates  the  Federal  CIO  Position 
Model  is  applicable  to  the  USAF.  The  model  is  intended  to  be  normative  and  to  produce 
a  visually  intuitive  depiction  of  the  CIO  position  that  indicates  that  the  twelve  roles  of  a 
Federal  CIO  operate  across  different  organizational  levels  of  an  organization  (Bernard, 
2001a).  Part  2  seeks  to  model  and  describe  the  CIO  position  from  an  organization 
management  theory  through  selection  of  programmed  choices  from  a  drop-down  menu. 

The  Part  3  questions  are  designed  to  answer  Research  Question  #1.  Also  apart  of 
Bernard’s  research,  this  survey  relies  upon  the  validated  Federal  CIO  Position  Evaluation 
Method  (FCPEM).  The  FCPEM  is  the  method  for  evaluating  whether  federal  agencies 
have  complied  with  the  intent  of  the  CCA  of  1996  as  they  established  CIO  positions.  The 
FCPEM  contains  thirteen  evaluation  criteria  and  was  tested  and  validated  by  Bernard 
through  key  actor  interviews  at  four  federal  agencies  and  focused  on  CIO  position 
establishment  activity  between  1996  and  2000.  This  is  a  unique  approach  in  CIO 
modeling,  representing  organizational  theory  and  CIO  duties  or  competency  areas.  The 
chief  reason  for  placing  the  CIO  competency  areas  in  the  context  of  an  organizational 
structure  is  to  reveal  more  about  how  and  where  these  competency  areas  function  in  the 
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complex  federal  agency  organization  (Bernard,  2001a).  Furthering  Bernard’s  findings, 
this  research  is  undertaken  to  “...replicate  finding[s]  in  other  agencies  and  to  further 
validate  the  use  of  FCPEM  in  conducting  this  type  of  public  policy  inquiry”  (Bernard, 
2001a).  Part  3  analyzes  a  matrix  of  CIO  Role  Evaluation  Standards  to  CIO  competencies 
again  by  selective,  programmed  choices.  Part  3  explores  normative  models  created  for 
the  federal  agency  CIO  position  and  it’s  roles. 

Both  the  Federal  CIO  Position  Model  and  the  FCPEM  have  been  updated  to 
reflect  the  changes  to  the  Federal  CIO  Core  Competencies  (2000).  This  research  makes 
use  of  the  enhanced  FCPEM  recommended  by  Bernard.  Questions  were  intended  to  elicit 
candid  answers  from  the  respondent  of  their  own  observations  of  how  each  of  the  topic 
areas  have  changed  the  USAF  since  the  implementation  of  the  CCA,  EO  13011,  and  the 
office  of  the  AF-CIO. 

Pilot  Testing 

Pilot  testing  was  conducted  with  five  active  duty  USAF  graduate  students 
enrolled  in  the  Information  Resource  Management  (G1R-02M)  program,  CIO  Track  at 
the  Air  Force  Institute  of  Technology.  Results  were  used  to  identify  any  questions  in  the 
survey  that  would  be  misleading,  redundant,  dichotomous,  or  would  otherwise  cause 
measurement  error.  Modifications  were  made  based  on  the  judgments  and 
recommendations  of  these  experts.  After  considerable  scoping,  a  viable  instrument  was 
produced  which  generated  reasonable  and  acceptable  results,  thereby  achieving  the 
intermediate  purpose  of  the  data  collection  effort  for  this  study. 
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Permission  to  Conduct  Research 


The  AF-CIO  office.  Pentagon,  Washington  D.C.,  sponsored  this  research.  The 
Air  Force  Survey  Branch  at  the  Air  Force  Personnel  Center  (HQ  AFPC/DPSAS) 
approves  all  surveys  that  are  administered  to  active-duty  Air  Force  personnel  without 
specific  commander  consent.  Once  the  survey  was  developed,  it  was  provided  along  with 
justification  for  the  survey  to  the  Air  Force  Survey  Branch.  It  was  approved  on  27 
December  2001  with  an  AFPC  Survey  Control  Number  of  USAF  SCN  02-001,  and  an 
expiration  of  31  May  2002.  Air  Force  Instruction  36-2601  governs  Air  Force  survey 
procedures.  The  researcher  adhered  to  the  AFI.  Additionally,  on  28  December  2001,  Dr. 
Bernard  gave  permission  and  support  to  both  modify  and  utilize  the  Federal  CIO  Position 
Model  and  the  FCPEM  survey  guide  he  developed  from  his  PhD  dissertation 


Selection  of  Sample  Size 

The  reliability  of  the  collected  data  is  dependent  on  the  size  of  the  sample,  not  the 
size  of  the  population  or  the  number  of  samples  solicited  (Alreck  and  Settle,  1995:30).  A 
power  analysis  was  completed  to  determine  the  required  sample  size  utilizing  the 
following  formula  (Department  of  the  Air  Force,  A  Guide  for  the  Development  of  the 
Attitude  and  Opinion  Survey,  1974:14-16); 


_ N(Z^)p(l-p) 

(N-l)(d')  +  (z-)p(l-p) 


where:  n  =  sample  size 
N  =  population 

p  =  maximum  sample  size  factor  (.5) 
d  =  desired  tolerance  (.10) 

z  =  factor  of  assurance;  1.282  for  a  90  percent  confidence  interval 
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Applying  the  formula  to  the  data  for  this  research  effort,  the  following  n  was 


determined. 


n  =  35 


249tl.282^').5tl-.5t 
(249-l)(.10^)  +  (1.282^).5(l-.5) 


The  power  analysis  of  .10  (90%)  revealed  that  35  returned  surveys  were  needed 
for  this  study  based  on  a  population  of  249. 

A  second  method  of  determining  necessary  sample  size  shows  that  a  minimum  of 
100  and  maximum  of  10%  of  the  population  is  considered  appropriate  parameters  for  a 
survey/questionnaire  (Alreck  &  Settle,  1995:62).  In  order  to  achieve  statistical  power, 
the  sample  size  was  set  to  attain  a  respondent  pool  of  at  least  35. 


Survey  Administration 

Survey  notification  was  made  by  e-mail  using  the  AF’s  standard  e-mail  naming 
convention  of  firstname.lastname@airforcebase.af.mil.  Addresses  were  generated  from 
the  list  of  personnel  received  from  the  AFIT  Registrar’s  office  and  sent  from  an  e-mail 
account  created  specifically  for  this  research.  To  avoid  the  potential  response  bias  of  a 
person  recognizing  the  name  of  the  researcher,  a  personal  e-mail  account  was  not  used. 
The  new  account  was  created  with  the  address  AFClQ.survev@,afit.edu.  All  e-mail 
notification  failures  were  delivered  to  this  account  and  monitored  by  the  researcher.  The 
text  of  the  notification  message  explained  that  the  survey  was  being  conducted  by  the  Air 
Force  Institute  of  Technology  to  evaluate  the  impact  of  the  Clinger-Cohen  Act  of  1996  on 
the  AF-CIO  and  the  USAF.  It  also  stated  the  research  was  sponsored  by  the  AF-CIO 
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Office  and  that  respondents  had  been  chosen  because  of  the  CIO  relate  experience  and 
expertise  as  an  IT/IRM  leader  in  the  USAF  The  web-based  survey  was  hosted  on  an 
AT IT  web  server  at  the  address  http://en.afit.edu/env/af-cio. 

Data  Analysis 

Stepwise  Analysis 

The  analytic  procedure  used  in  this  study  consists  of  four  major  steps: 

1)  Stratified  sampling  occurs  in  the  first  step.  This  entails  separating  the 
sampling  frame  into  non-overlapping  groups  and  taking  a  sample  from  each  one.  The 
individual  respondents  are  classified  into  2  independent  and  mutually  exclusive  groups 
based  on  responses  to  the  background  or  demographic  questions. 

2)  In  the  seeond  step  individual  responses  to  the  self-administered  survey  on  the 
groups  of  “outeomes”  questions  are  examined  utilizing  descriptive  statistical  methods.  A 
content  analysis  methodology  is  also  used  to  explore  responses  to  the  open-ended 
questions.  This  is  an  important  step  as  it  helps  to  answer  the  research  question  of  how  the 
USAF  has  changed  since  the  passage  of  the  CCA  and  the  creation  of  the  AF-CIO  office. 

It  also  relates  to  substantive  complianee  with  the  CCA. 

3)  In  the  third  step  the  Federal  CIO  Position  Model  is  tailored  for  the  AF-ClO 
office,  again  based  on  the  respondents  answers.  The  descriptive  model  provides  a 
visually  intuitive  depiction  of  the  AF-CIO  position  that  indicates  the  12  roles  of  a  Federal 
CIO  that  operate  across  different  levels  of  the  organization. 

4)  In  the  fourth  step  the  enhanced  Federal  CIO  Position  Evaluation  Method 
(FCPEM)  is  used  to  relate  what  Bernard  termed  the  CIO  Role  Evaluation  Standards,  to 
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comply  with  the  Clinger-Cohen  Act  (CCA).  These  role  evaluation  standards  relate 
directly  to  specific  mandates  of  the  CIO  from  the  CCA  and  were  validated  by  Bernard’s 
research.  The  FCPEM  is  used  to  examine  if  the  USAF  is  in  compliance  with  the  CCA. 

A  statistical  analysis  software  application  tool  called  JMP  IN,  version  4.0,  from 
the  SAS  Institute  Inc.,  was  specifically  used  to  run  the  Fisher-lrwin  Exactness  test  for 
differences  in  proportions  between  two  populations. 

Qualitative  Content  Analysis 

A  content  analysis  of  the  responses  to  open  ended  questions  from  the  web-based 
survey  is  used  to  in  order  to  explore  the  population  under  study.  A  concept  is  chosen  for 
examination  and  the  analysis  involves  quantifying  and  qualifying  its  presence. 

The  researcher  follows  the  eight-step  process  indicated  in  the  Colorado  State 
University  guide  on  content/relational  analysis  (Content  Analysis,  2001),  which  are; 

1)  Identify  the  question, 

2)  Choose  a  sample  or  samples  for  analysis, 

3)  Determine  the  type  of  analysis, 

4)  Reduce  the  text  to  categories  and  code  for  words  or  patterns, 

5)  Explore  the  relationships  between  concepts  (Strength,  Sign  &  Direction), 

6)  Code  the  relationships, 

7)  Perform  analyses,  and 

8)  Map  out  the  representations 

As  a  content  analysis  researcher  commented,  “There  is  no  simple  right  way  to  do 
content  analysis.  Instead,  investigators  must  judge  what  methods  are  most  appropriate 
for  their  substantive  problems”  (Weber,  1990:13).  The  task  of  the  content  analysis  for 
this  research  begins  with  identifying  concepts  present  in  a  given  text  or  set  of  texts. 
Flowever,  using  a  subset  of  concept  analysis  called  relational  analysis,  the  researcher, 

“. .  .goes  beyond  the  presence  of  the  words  by  exploring  the  relationships  between  the 
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concepts  identified”  (Concept  Analysis,  2001).  The  focus  of  relational  analysis  is  to  look 
for  semantic  or  meaningful  relationships  among  the  words.  The  researcher  explores  the 
text  to  identify  if  is  there  is  a  stronger  presence  of  positive  or  negative  words  used  with 
respect  to  the  research  questions. 

Quantitative  Statistical  Analysis 

Fisher- Irwin  Exactness  Test 

The  Fisher- Irwin  test  is  the  most  powerful  test  of  equality  of  two  proportions  if 
the  random  variable  of  interest  is  qualitative  and  dichotomous  (Marascuilo  and 
McSweeny,  1977:  96).  The  Fisher-lrwin  procedure  tests  the  hypothesis  of  equality  of 
population  proportions: 

Ho:  Pi  =  ?2 

for  two  independent  groups  that  are  classified  dichotomously  on  an  outcome  measure 
(Ibid.).  The  test  is  used  to  determine  whether  the  probability  parameters  of  two 
independent  binomially  distributed  random  variables  are  equal.  The  hypergeometric 
distribution  is  used  for  calculating  probabilities  for  samples  drawn  from  relatively  small 
populations  and  without  replacement  (Devore,  2000:128).  Although  the  hypergeometric 
distribution  will  ultimately  be  used  in  testing  the  equality  of  two  population  proportions, 
the  distribution  itself  is  presented  in  terms  of  a  single  population  (Ibid). 

A  classification  of  the  respondents  resulted  in  dividing  the  sample  population  into 
two  mutually  exclusive  and  independent  groups  -  Executive  Level  and  Managerial  Level. 
The  Executive  Level  group  includes  AF-CIO  staff,  members  of  the  AF/SC  staff,  and 
EXCOM  members,  including  all  USAF  Major  Command  (MAJCOM)  CIOs  and 
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Headquarters  Air  Force  (HAF)  Functional  CIO  representatives.  The  Managerial  Level 
group  includes  base-level  individuals  who  are  responsible  for  CIO  related  duties,  such  as 
USAF  Communications  Squadron  Commanders  and  Group  Commanders. 

This  research  requires  the  use  of  the  Fisher-Irwin  test  to  determine  whether  the 
distribution  of  the  independent  group  responses  can  be  expressed  as  a  single  population. 

A  test  of  proportions  between  populations  is  used  to  determine  if  there  is  a  statistically 
significant  difference  between  the  Executive  Level  and  the  Managerial  Level  groups. 

The  null  hypothesis  for  this  study  establishes  that  there  is  no  difference  between  the 
executive  and  managerial  groups  with  regard  to  the  questions  in  Part  1  of  the  survey.  If  a 
difference  exists,  we  must  reject  the  null  hypothesis  and  declare  the  groups  as 
significantly  different. 

Summary 

This  research  polls  individuals  who  perform  CIO  tasks  mandated  by  the  CCA.  A 
web-based  survey  was  created  to  assist  in  answering  the  research  questions.  The 
notification  to  participate  in  the  survey  was  e-mailed  to  every  individual  in  the 
population.  Statistical  analysis  is  conducted  to  infer  differences  between  populations. 
Content  analysis  is  conducted  to  surmise  and  explain  the  results. 

The  researcher  gathered  relevant  information  on  public  and  private  sector  research 
regarding  CIOs.  Documents  on  the  impacts  of  federal  IRM  policy  on  federal  government 
agencies  were  reviewed.  Interviews  were  conducted  with  notable  individuals  who  have 
influenced  the  field  of  federal  IRM  policy  and  management.  The  results  of  this 
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information  gathering  are  used  to  lay  the  foundation  for  this  grounded  theory  work  in 
order  to  provide  the  structure  required  to  explain  and  answer  the  research  questions. 

The  impact  of  CIO  mandates  of  the  CCA  were  explored  in  this  research  through: 
1)  an  analysis  of  the  language  of  the  CCA  and  other  IRM-related  federal  legislation,  2)  a 
review  of  information  gained  through  the  self  administered,  web-based  survey  results  of 
USAF  members,  3)  information  gained  from  documentation  of  public  sector  CIOs  and 
their  experiences,  and  4)  research  conducted  to  bridge  the  gap  between  organizational 
and  policy  theory,  and  federal  agency  CIOs. 

This  chapter  described  a  research  method  to  investigate  the  impacts  of  the  CCA 
on  the  USAF.  It  described  the  survey  methodology,  content  analysis  methodology,  and 
grounded  theory  methodology  along  with  the  operationalized  constructs.  Finally,  this 
chapter  described  how  the  collected  data  was  analyzed.  The  results  of  the  analysis  are 
presented  in  Chapter  IV.  The  interpretation  and  findings  of  this  multi-modal  study,  along 
with  recommendations  for  future  research  efforts,  are  presented  in  Chapter  V. 
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IV.  Results  and  Analysis 


Overview 

This  chapter  provides  the  overall  results  of  the  data  collection  effort.  As  stated 
previously,  the  purpose  of  this  research  is  to  examine  the  impact  of  the  Clinger-Cohen 
Act  of  1996  (CCA)  on  the  AF-CIO  office  and  the  USAF.  The  investigative  questions  to 
be  answered  are: 

1)  Are  the  USAF  implementation  efforts  consistent  with  the  intent  of  the  CCA? 

2)  To  what  extent  can  a  model  of  the  AF-CIO  position  illustrate  the  level  of  the 
AF-CIO  office  implementation  efforts? 

3)  What  outcomes  can  be  found  as  a  result  of  the  CCA  and  the  AF-CIO  office 
requisite  to; 

A)  The  Federal  CIO  Core  Competencies, 

B)  The  performance  based  aspects  of  the  CCA,  Section  5123, 

C)  Key  IRM  achievement  areas  from  the  public  and  the  private  sector, 

D)  The  roles  and  responsibilities  of  the  AF-CIO. 

4)  Have  changes  in  USAF  operations  occurred  as  a  result  of  instituting  an  AF-CIO 
office? 

5)  How  have  these  changes  impacted  the  USAF? 

In  order  to  answer  these  questions  this  chapter  will  present  the  analytic  procedure 
used  for  this  study.  First,  an  analysis  of  the  response  rate  is  presented.  Next,  the 
demographic  data  of  the  survey  participants  is  offered.  Finally,  an  analysis  of  responses 
from  each  part  and  section  from  the  web-based  survey  is  given.  Screenshots  of  each  page 
of  the  web-based  survey  are  provided  in  Appendix  F. 
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Response  Rates 

This  research  polled  the  entire  population  of  interest  consisting  of  249  people.  A 
notification  e-mail  message  to  participate  in  the  web-based  survey  was  sent  to  all  249 
potential  participants.  However,  84  e-mail  messages  were  rejected  due  to  faulty  e-mail 
addresses.  The  researcher  was  able  to  recover  39  of  the  84  faulty  e-mail  addresses. 
Essentially,  204  successful  e-mail  notifications  to  potential  respondents  (requests  to 
participate  in  the  research  survey)  were  made.  Two  follow-up  e-mailings  were  made  to 
some  of  the  participants  to  encourage  response.  Follow-up  e-mail  notifications  were  not 
made  to  EXCOM  members,  AF/SC  staff,  and  C33S4  General  Officers  out  of  respect  to 
the  positions  they  hold  and  the  level  of  responsibility  they  bear  across  the  entire  USAF. 

158  of  the  204  candidates  participated  in  the  research  survey  for  a  response  rate  of 
77.5%.  Table  15  below  lists  the  response  rate  results  broken  down  into  the  applicable 
parts  and  seetions  of  the  survey  instrument. 


Table  15:  Response  Rates  for  all  Respondents 


Survey  Sections 

IViimher  nf 
Respondents 

Resnnnse 

Rate 

Rarkffrniind 

158 

77.5% 

Part  la  -  Competencies 

100 

49.0% 

Part  lb  -  Performance 

87 

42.6% 

Part  Ic  -  Outcomes 

78 

38.2% 

Part  Id  -  Roles 

72 

35.3% 

Part  2  -  FCPM 

66 

32.4% 

Part  3a  -  FCPEM 

53 

26.0% 

Part  3b  -  FCPEM 

53 

26.0% 

Roughly  30%  of  the  individuals  beginning  the  survey  actually  completed  it. 
Some  replies  were  discarded  due  to  ineomplete  responses. 
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Stratification  of  Data 


The  intent  of  the  research  was  to  poll  the  entire  population  of  249  individuals,  and 
stratify  them  as  either  the  Executive  Level  of  the  Managerial.  However,  due  to  a  problem 
with  the  data  collection  effort  only  38%  of  the  total  data  collected  could  be  stratified. 
Therefore,  results  will  be  reported  by  strata  where  possible,  as  well  as  unstratified  across 
all  respondents.  Doing  this  resulted  in  creating  and  assessing  four  groups.  The  Executive 
Level  group  contains  all  those  respondents  known  to  be  at  the  Executive  Level  (n=9). 

The  Managerial  Level  group  contains  all  those  respondents  known  to  be  at  the 
Managerial  Level  (n=50).  The  combination  of  the  Executive  Level  and  the  Managerial 
Level  will  be  called  the  Stratified  Group  (n=59).  All  Respondents  is  the  name  of  the 
group  that  contains  both  the  Stratified  Group  and  those  that  could  not  be  stratified 
(n=158).  The  table  below  summarizes  the  groups. 

Table  16:  Group  Summary 


Name  of  Group 

Total  Number  of 
Respondents 

Executive  Level 

9 

Managerial  Level 

50 

Stratified  Group 

59 

All  Respondents 

158 

Response  rates  for  the  stratified  data  are  indicated  in  Table  17  below. 
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Table  17:  Response  Rates  for  Stratified  Population 


Survey  Sections 

Niimhpr  nf 
Respondents 

Rate 

Rarkcrrnund 

SQ 

28  9»/n 

Part  la  -  Competencies 

53 

26.0% 

Part  lb  -  Performance 

47 

23.0% 

Part  Ic  -  Outcomes 

35 

17.2% 

Part  Id  -  Roles 

35 

17.2% 

Part  2  -  FCPM 

27 

13.2% 

Part  3a  -  FCPEM 

25 

12.3% 

Part  3b  -  FCPEM 

23 

11.3% 

Determining  agreement  between  these  groups  is  important  to  this  study  as  the 
results  are  used  to  explore  the  impacts  of  the  CCA  on  the  USAF  and  to  understand  the 
organizational  culture,  attitudes,  beliefs,  and  knowledge  of  each  group. 


Demographic  Information 

The  demographic  information  was  collected  in  such  a  way  as  to  protect  the 
anonymity  of  the  respondents  while  providing  a  means  to  stratify  between  groups  in  the 
population.  For  stratification  purposes,  the  population  was  separated  into  2  groups  - 
Executive  Level  and  Managerial  Level.  For  the  purpose  of  comparing  population 
responses,  a  Stratified  Level  group  and  an  All  Respondents  group  were  also  established. 
.  The  following  tables  express  the  summaries  for  each  background  question. 


Table  18:  Years  of  Experience  Summary 


1  Years  of  Experience  | 

Group 

Respondents 

Mean 

Standard  Dev. 

Std  Error 

Median 

Max 

Min 

Executive 

9 

26.44 

1.81 

0.60 

26 

30 

24 

Managerial 

50 

17.18 

5.77 

0.82 

18 

30 

4 

Stratified 

59 

18.59 

6.31 

0.82 

19 

30 

4 

All 

158 

16.95 

6.58 

0.52 

18 

30 

1 
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Table  19:  Years  in  Current  Position  Summary 


1  Years  in  Current  Position  | 

Group 

Respondents 

Mean 

Standard  Dev. 

Std  Error 

Median 

Max 

Min 

Executive 

9 

3 

1.73 

0.58 

3 

7 

1 

Managerial 

50 

1.76 

1.89 

0.27 

1 

10 

0.5 

Stratified 

59 

1.95 

1.90 

0.25 

2 

10 

0.5 

All 

158 

1.59 

1.46 

0.12 

1 

10 

0.5 

Table  20:  Years  in  Current  Organization  Summary 


1  Years  in  Current  On 

ganization  | 

Group 

Respondents 

Mean 

Standard  Dev. 

Std  Error 

Median 

Max 

Min 

Executive 

9 

2.44 

0.88 

0.29 

2 

4 

1 

Managerial 

50 

1.86 

1.91 

0.27 

1.5 

10 

0.5 

Stratified 

59 

1.95 

0.23 

0.23 

2 

10 

0.5 

All 

158 

1.83 

2.09 

0.17 

1 

20 

0.5 

Analysis  of  Survey  Responses 


Part  1:  Outcomes 

Part  1  of  the  web-based  survey  addresses  Research  Questions  #3,  #4,  and  #5. 
Analyses  of  Part  1  of  the  web-based  survey  are  described  through  statistical  analysis  and 
content  analysis 

Fisher-Irwin  Exactness  Test  Procedure 

The  Fisher-Irwin  Exactness  Test  was  conducted  between  the  4  groups  (Executive 
and  Managerial,  Stratified  and  All  Respondents)  to  check  for  equality  between  population 
proportions  of  each  groups’  answers.  The  results  for  each  of  4  groups  through  42 
different  questions  were  independently  loaded  into  JMP  IN®  to  determine  the  results  of 
the  Exact  Test. 
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Group 


Count 

Total  % 

Col% 

Row  % 

1 

2 

1 

8 

23 

31 

17.39 

mm 

67.39 

88.89 

62.16 

25.81 

74.19 

2 

1 

14 

15 

2.17 

30.43 

32.61 

11.11 

37.84 

6.67 

93.33 

9 

37 

46 

19.57 

80.43 

Left 

Right 

2-Tail 

0.9817 

0.1257 

0.2348 

_ 

Figure  8:  Results  of  Fisher-Irwin  Exactness  Test 
Using  an  alpha  value  of  .05,  this  p-value  result  of  .2348  indicates  we  should  not 
reject  the  null  hypothesis  since  the  p-value  is  greater  than  the  alpha  value.  Therefore,  we 
fail  to  reject  the  null  hypothesis  that  there  is  no  statistical  difference  between  these  two 
groups  with  respect  to  the  responses  for  this  question.  The  distribution  of  the 
independent  group  responses  can  be  expressed  as  a  single  population;  there  is  no 
statistical  difference  between  the  Executive  and  Managerial  Level. 

Following  a  premise  of  “It  has  been  six  years  since  Congress  passed  the  Clinger- 
Cohen  Act,”  respondents  were  asked  “As  a  result  of  the  CCA  and  the  AF-CIO  office,  is 
the  USAF  different  today  in  these  specific  areas?”  Content  analysis  was  performed  to 
reveal  relationships  among  the  comments  and  to  determine  the  impact  areas.  The 
researcher  explored  the  comments  to  look  for  word  patterns,  identify  the  presence  of 
themes,  define  the  categories,  classify  text,  and  to  determine  the  impact  areas.  IRM 
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graduate  students  at  the  Air  Force  Institute  of  Technology  and  various  members  of  the 
researchers  thesis  committee  verified  the  impact  areas. 

Section  la,  Federal  CIO  Core  Competencies 

Respondents  were  asked  if  they  believed  the  USAF  was  different  today  as  a  result 
of  the  Clinger-Cohen  Act  and  the  AF-CIO  office,  with  respect  to  the  12  Federal  CIO 
Core  Competencies.  A  total  of  100  individuals  responded  to  this  series  of  questions. 
Nearly  300  comments  were  received. 

The  following  table  shows  the  results  of  the  Fisher-lrwin  Exactness  Test 
regarding  the  Federal  CIO  Core  Competencies  2000.  In  1 1  of  the  12  competency  areas, 
there  is  no  significant  difference  between  the  Executive  Level  and  the  Managerial  Level 
groups.  The  corresponding  p-value  for  the  core  competency  question  regarding  Project 
and  Program  Management  is  .02,  leading  to  the  rejection  of  the  null  hypothesis. 

Table  21:  Stratified  Level  Test  Results  for  Federal  CIO  Competencies  2000 


1  Fisher-lrwin  Exactness  Test  Results  (a  =  .05)  | 

Federal  CIO  Core  Competencies  (2000) 

Executive 

Level 

(n=9) 

Managerial 
Level  (n=44) 

p  value 

IR  Strategy  &  Planning 

5 

24 

1.00 

E-Gov  /  E-Bus  /  E-Comm 

6 

28 

1.00 

IT  Security  /  Information  Assurance 

6 

25 

0.72 

Leadership  /  Managerial 

7 

27 

0.46 

Policy  /  Organizational 

7 

25 

0.29 

Acquisition 

4 

11 

0.25 

Technical 

4 

11 

0.25 

Desktop  Technology  Tools 

4 

10 

0.22 

Capital  Planning  &  Investment  Assessment 

4 

9 

0.20 

Performance  Assessment:  Models  and 

4 

8 

0.18 

Process  /  Change  Management 

7 

18 

0.07 

Project  /  Program  Management 

5 

7 

0.02 
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An  analysis  was  also  conducted  to  test  equality  between  proportions  of  the 
Stratified  Level  group  and  the  All  Respondents  group  in  the  area  of  the  Federal  CIO  Core 
Competencies.  Again,  the  Fisher- Irwin  test  was  used.  The  following  results  show  that 
there  is  no  signifieant  differenee  between  the  two  groups. 


Table  22:  Stratified  vs.  All  Respondents  Test  Results  for  Federal  CIO  Competencies  2000 


Fisher-Ii*win  Exactness  Test  Results  (a  -  .05) 


Federal  CIO  Core  Competencies  (2000) 

Stratified 

Level 

(n=53) 

All 

Respondents 

(n=100) 

p  value 

IR  Strategy  &  Planning 

29 

52 

0.87 

E-Gov  /  E-Bus  /  E-Comm 

34 

57 

0.49 

IT  Seeurity  /  Information  Assurance 

31 

59 

1.00 

Eeadership  /  Managerial 

34 

60 

0.73 

Poliey  /  Organizational 

32 

60 

1.00 

Aequisition 

15 

40 

0.16 

Teehnieal 

15 

32 

0.71 

Desktop  Teehnology  Tools 

14 

35 

0.36 

Capital  Planning  &  Investment  Assessment 

13 

35 

0.20 

Performanee  Assessment:  Models  and 

12 

29 

0.45 

Process  /  Change  Management 

25 

42 

0.61 

Project  /  Program  Management 

12 

24 

1.00 

The  content  analysis  established  48  impact  areas.  The  following  table  reports  the 
results  of  the  most  frequent  responses.  The  entire  response  list  can  be  viewed  in 
Appendix  G. 
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Table  23:  Reported  Changes  to  the  USAF  Due  to  the  Federal  CIO  Core  Competencies 


Part  la,  Impact  from 

Core  Competencies 

#of 

Comments 

Standardized  Approach 

19 

Working  Groups  at  Base/MAJCOM  Level 

15 

Centralized  Management  of  Networks 

12 

Responsive  Organizational  Structure 

11 

IT  Enterprise  Policy  Awareness 

10 

Business  Case  Development 

8 

Information  as  a  Strategic  Resource 

8 

CIO  Mandated  by  Law 

7 

Consolidation  of  Neworks/Servers 

7 

Development  of  TCO 

7 

More  Clear  Direction 

7 

Network  Centric  Awareness 

6 

IT  Initiatives 

5 

Strategic  Information  Approach 

5 

Architecture  Office  Established 

4 

Better  Performance  Measures 

4 

Config  Control  AF-wide 

4 

Enterprise  Strategy 

4 

Network  Consolidation 

4 

Timely  Acquisition 

4 

Based  on  the  results  of  the  analysis,  the  impact  areas  are  reported  as  a  single  set  of 
impacts  specific  to  the  Federal  CIO  Core  Competencies.  However,  since  the  analysis 
shows  a  difference  in  responses  regarding  the  Competency  of  Project  and  Program 
Management  question,  the  results  are  reported  separately  in  the  table  below. 

Table  24:  Impact  Area  Responses  for  Project  /  Program  Management 


1  Project  /  Program  Management  | 

1  Executive  Level  | 

Managerial  Level 

Unstratifled  Group 

n 

Network  Centric  Awareness 

Business  Case  Development 

More  Clear  Direction 

m 

Strategic  Information  Approach 

Standardized  Approach 

SPO  Overhead  too  Costly 

m 

Consolidation  of  Neworks 

Centralized  Management 

Working  Groups  at  Base/MAJCOM 

Responsive  and  Lean  Org  Structure 

Responsive  Org  Structure 

IT  Enterprise  Policy  Awareness 

m 

Working  Groups  at  Base/MAJCOM 

Development  of  TCO 

m 

Untimely  Acquisition  Process 

PPBS  Hasn’t  Changed  Enough 

m 

Non-standardized  Approach 

H 
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Section  lb,  Performance 

Respondents  were  asked  if  they  believed  the  USAF  was  different  today  as  a  result 
of  the  Clinger-Cohen  Act  and  the  AF-CIO  office,  with  respect  to  Section  5123, 
Performance  and  Results  Based  Management,  of  the  Clinger-Cohen  Act  of  1996. 

Table  25  shows  the  results  of  the  Fisher-lrwin  Exactness  Test  of  equality  of 
population  proportions  regarding  the  performance  aspects  of  the  CCA,  section  5123. 
None  of  the  null  hypotheses  were  rejected  in  this  area.  These  results  suggest  there  is  no 
difference  between  the  groups. 

Table  25:  Stratified  Level  Test  Results  For  Performance 


Fisher-lrwin  Exactness  Test  Results  (a  =  .05) 

Performance  (CCA  Section  5123) 

Executive 

Level 

(n=9) 

Managerial 

Level 

(n=38) 

p  value 

Delivering  Services 

Ensuring  Perfomance  Measures  Support 
Establishing  Goals 

Qualitative  Benchmarking 

Ensuring  Prescribed  Performance  Measures 
Analyzing  Air  Force  Missions 

Ensuring  Adequate  Information  Security  P^ 
Preparing  Annual  Report  to  Congress 

3 

17 

1.00 

2 

12 

0.70 

5 

16 

0.49 

4 

10 

0.42 

5 

13 

0.27 

2 

18 

0.26 

8 

23 

0.23 

4 

6 

0.08 

Note:  =  Policy,  Procedure,  Performance 


An  analysis  was  also  conducted  to  compare  the  results  of  the  Stratified  Level 
group  against  those  of  All  Respondents  in  the  area  of  Section  5123  of  the  CCA.  There 
was  no  significant  difference  between  the  two  groups. 
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The  content  analysis  established  65  impact  areas  for  this  section.  Table  26  reports 
the  results  of  the  most  frequent  responses.  The  entire  response  list  can  be  viewed  in 
Appendix  G.  Based  on  the  results  of  the  statistical  analysis,  the  impact  areas  are  reported 
as  a  single  set  of  impacts 

Table  26:  Reported  Changes  to  the  USAF  Due  to  the  Performance  Aspects  of  the  CCA 


Part  lb,  Impact  from 

CCA,  Section  5123 

#of 

Comments 

Use  of  Business  Models 

9 

IT  Initiatives 

8 

Too  Soon  to  Tell 

8 

High  Level  Support 

7 

Working  Groups  at  Base/M AJCOM  Level 

5 

Enterprise  Solution 

4 

IT  Enterprise  Policy  Awareness 

4 

Little  Change  to  PPBS 

4 

Accountability  Greater 

3 

CIO  Leadership 

3 

Equipment  and  Config  Control  Better 

3 

Responsive  Org  Structure 

3 

Strategic  Planning 

3 

TCO 

3 

Section  Ic,  Outcomes  and  Outputs 

Respondents  were  asked  if  they  believed  the  USAF  had  achieved  significant 
outcomes  relative  to  Key  IRM  Achievement  Areas  identified  in  previous  IRM/IT 
research  and  reports,  and  found  in  the  Chapter  2.  This  section  asked  18  questions.  The 
respondents  answered  either  yes  or  no.  Individual  responses  were  requested  for  each  of 
the  questions.  The  respondents  were  also  requested  to  explain  their  answers.  A  total  of 
78  individuals  responded  to  this  series  of  questions.  Over  300  comments  were  received 
The  following  table  shows  the  results  of  the  Fisher- Irwin  Exactness  Test  of 
equality  of  population  proportions  regarding  key  IRM  achievement  areas  from  public  and 
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the  private  sector  research  and  reports.  It  shows  there  is  no  difference  between  the 
groups  on  most  items.  However,  the  p-value  for  the  key  IRM  area  of  Budget  Requests  is 
.0033.  This  signifies  that  the  null  hypothesis  is  rejected  for  this  item. 


Table  27:  Stratified  Level  Test  Results  for  Outcomes  and  Outputs 


Fisher-Irwin  Exactness  Test  Results  (a  =  .05) 

Outcomes  and  Outputs 
(previous  research) 

Executive 

Level 

(n-9) 

Managerial 
Level  (n=26) 

p  value 

Compliance  Standards 

4 

10 

1.00 

Strategic  Planning 

5 

15 

1.00 

Investments 

3 

9 

1.00 

Customer  Satisfaction 

2 

8 

1.00 

Lifecycle  Maintenance 

2 

5 

1.00 

IT  Reliability 

4 

10 

1.00 

Enterprise  Solution 

5 

15 

1.00 

USAF  Operations 

5 

12 

0.71 

IT  Usability 

5 

11 

0.70 

National  Security  Systems 

3 

7 

0.69 

Business  Processes 

2 

8 

0.69 

Architecture 

6 

13 

0.46 

IT  Availability 

3 

13 

0.46 

Performance  Measurements 

3 

5 

0.40 

Deployment  of  Services 

5 

8 

0.24 

Contingency  Preparedness 

3 

3 

0.16 

Disclosure  of  Costs 

4 

4 

0.16 

Budget  Requests 

6 

3 

0.00 

An  analysis  was  also  conducted  to  compare  the  results  of  the  Stratified  Level 
group  against  those  of  All  Respondents  in  the  area  of  key  IRM  achievement  areas  from 
public  and  the  private  sector  research  and  reports.  Again,  the  Fisher- Irwin  test  was  used. 
There  is  no  significant  difference  between  the  two  groups. 
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The  content  analysis  established  63  impact  areas.  Table  28  reports  the  results  of 


the  most  frequent  responses.  The  entire  response  list  can  be  viewed  in  Appendix  G. 


Table  28:  Reported  Changes  to  the  USAF  Due  to  Key  IRM  Achievement  Areas 


Part  Ic,  Impacts  from  Key 
IRM  Achievement  Areas 

#of 

Comments 

Realistic  to  IT,  Not  Ops 

25 

Too  Soon  to  Tell 

24 

Process  Not  Understood 

19 

High  Level  Support 

16 

TQM  Culture  Hinders 

15 

Goals  tied  to  Money  and  Staffing 

12 

Perfomance  Oriented 

12 

Little  Change  to  PPBS 

11 

Using  Business  Models 

10 

Slow  Implementation 

9 

Network  Centric  Ops  Support 

9 

Stale  Metrics 

5 

Enterprise  Approach 

5 

NOSC  is  Key 

5 

Responsive  Org  Structure 

5 

No  Money 

4 

Not  Reactive 

4 

AF  Portal 

4 

IT  Initiatives 

4 

Stovepiped 

3 

Based  on  the  results  of  the  analysis,  the  impact  areas  are  reported  as  a  single  set  of 
impacts  specific  to  the  Key  IRM  Achievement  Areas.  However,  since  the  statistical 
shows  a  difference  exists  in  the  responses  regarding  the  Budget  Requests  question  the 
results  are  reported  separately  in  the  table  below. 
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Table  29:  Impact  Area  Responses  for  Budget  Requests 


1  Budget  Requests  | 

1  Executive  Level 

Managerial  Level 

Unstratified  Group 

1 

CIO  Effective 

Budget  Requests  not  Successful 

Realistic  to  IT  not  Ops 

2 

No  Authority 

CIO  Office  not  Effective 

Stovepiped 

B 

Politics  with  Functionals 

Using  Business  Models 

Perfomance  Oriented 

0 

High  Level  Support 

B 

NOSC  is  Key 

B 

Slow  Implementation 

7 

Process  Not  Understood 

8 

Little  Change  to  PPBS 

B 

No  IT  Priorities 

Section  Id,  CIO  Roles 

The  CIO  Roles  seetion  analyzes  the  possibility  of  identifying  appropriate  roles 
and  responsibilities  of  the  CIO.  The  stratified  responses  explore  Executive  Level  and 
Managerial  Level  agreement  for  the  roles  specified  in  Appendix  C,  “MAJCOM  CIOs  and 
HAL  Functional  CIO  Representatives  Roles  and  Responsibilities,  2  January  2002.” 
Strength  areas  and  shortfalls  in  this  area  are  identified. 

The  survey  did  not  ask  respondents  to  rank  their  roles  in  reference  to  those 
included  in  Appendix  C.  However,  a  content  analysis  of  the  responses  revealed  some 
roles  were  selected  more  frequently  than  others.  The  following  table  lists  the  results. 
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Table  30:  CIO  Roles  Selection  for  All  Respondents 


CIO  Roles  and  Responsibilities 

#  Times 

Selected 

(n=72) 

Selection 

Rate 

Capital  Planning  and  Investment  Control; 

55 

76.4% 

Information  Assurance: 

53 

73.6% 

Technology  Assessment: 

51 

70.8% 

Standards  &  Architecture: 

50 

69.4% 

Training  and  Education: 

49 

68.1% 

Strategic  Planning: 

48 

66.7% 

Information  Technology  Acquisition: 

47 

65.3% 

Process  Improvement: 

46 

63.9% 

Performance  Measures: 

43 

59.7% 

E-Government/E-Bus  iness: 

42 

58.3% 

Information  and  Knowledge  Management: 

40 

55.6% 

Asked  about  the  appropriateness  of  these  same  roles,  nearly  85%  of  respondents 
agreed  the  listed  roles  were  appropriate  for  their  level.  A  majority  of  those  who  did  not 
agree  eited  that  Standards  and  Architecture  belonged  at  the  Executive  Level.  Also,  those 
at  the  Managerial  Level  felt  while  the  roles  were  important,  the  roles  at  their  level  should 
be  focused  on  technology  and  providing  operations  capability  with  the  best  equipment 
technology  and  availability.  Both  the  Managerial  Level  and  the  Executive  Levels  agreed 
that  strategic  planning  should  be  directed  from  the  top  and  implemented  at  the  base  and 
functional  level. 

Over  80%  responded  that  current  roles  and  responsibilities  should  remain  intact, 
without  adding  or  eliminating  any  at  this  time.  A  majority  of  the  rest  proposed  that 
process  improvement,  quality  control,  and  metrics  should  be  eliminated  from  the 
Managerial  Level  role.  There  was  no  further  explanation.  However,  Managerial  Level 
respondents  also  stated  Budget  Controls  should  be  initiated  at  their  level  in  order  to 
reduce  redundant  systems  and  to  realize  the  products  they  budget  years  in  advance  for 
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instead  of  using  the  funds  for  other  priorities.  By  far,  the  greatest  number  of  comments 
received  was  in  answer  to  the  question  regarding  “shortfalls.”  The  following  table 
summarizes  the  shortfalls  from  all  respondents. 

Table  31:  CIO  Roles  Shortfalls  (All  Respondents) 


Shortfalls  Identified 

#  Times 

Cited 

(n-72) 

Selection  Rate 

Manpower  and  Staffing 

34 

MBM 

Funding 

24 

mssm 

Feadership  Priorities 

17 

23.6% 

Empowerment  /  Ownership  of  Mission 

16 

22.2% 

Training 

15 

20.8% 

Workload  Tempo 

13 

18.1% 

Controlling  Changes 

7 

9.7% 

Equipment  /  Technology 

7 

9.7% 

No  Vision  into  Base-level  and  Functional  Spending 

7 

9.7% 

Under  Leadership  Priorities,  respondents  stated  that  in  the  recent  past,  with  the 
new  IT  initiatives,  everything  has  been  a  priority.  This  is  very  similar  to  the  Controlling 
Changes  shortfall  area  except  that  this  area  appears  to  be  originating  from  within  the 
Managerial  Level  only.  Collateral  effects  are  experienced  from  Manpower  and  Staffing 
shortages.  Under  Empowerment/Ownership  of  Mission,  respondents  cited  they  have  no 
authority,  or  they  lack  influence,  when  providing  a  business  case  at  their  level. 
Frequently,  operations  and  functional  organization  leaders  impose  their  own  technical 
solutions,  suboptimizing  resources  without  regard  to  the  implications  of  buying, 
integrating,  and  maintaining  the  implemented  solution. 
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Changes  in  the  USAF  Since  the  CCA 

The  open-ended  questions  in  Part  1  posed  the  question  in  the  form  “what  changes 
have  occurred  as  a  result  of  the  CCA  and  the  AF-CIO. . for  specific  areas.  Therefore, 
the  perceived  changes  are  specifically  attributed  to  the  CCA  and  the  AF-ClO  office.  The 
following  table  summarizes  the  changes  that  were  indicated  from  the  content  analysis. 
While  specific  mission  operations  were  not  generally  mentioned,  the  comments  did  cover 
support  operations  specific  to  IT  and  IRM. 
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Table  32:  Sample  Responses  Relating  to  Changes  Made  in  the  USAF  Since  the  CCA 


Impacts  to  the  USAF  as  a 
Result  of  the  CCA 

Too  Soon  to  Tell _ 

Business  Case 
Development  /  Models 

IT  Initiatives 
Realistic  to  IT,  Not  Ops 


Support  From  High  Levels 

Working  Groups  at 
Base/MAJCOM  Level 
Not  All  Process  are 
Understood _ 

Standardized  Approach 


Little  Change  to  PPBS 

Network  Centric  Awareness 

/  Ops  Support _ 

TQM  Culture  Hinders 

Results _ 

IT  Enterprise  Policy 
Awareness _ 

Goals  tied  to  Money  and 
Staffing 

Perfomance  Oriented 

Measurements _ 

Responsive  Organizational 
Structure _ 

CIO  Providing  Leadership 

Information  as  a  Strategic 
Resource _ 

Development  of  Total  Cost 
of  Ownership 


More  Clear  Direction 
Enterprise  Solution 


#  of 

Comments 
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Sample  Response 


Processes  are  still  in  development. 


Change  in  focus  streamlined  AF  networks  to  behave  as  IT 
business  entities,  rather  than  simply  support  organizations. 


IT  initiatives  and  the  famous  west  -coast  meetings  really 
opened  our  eyes. _ 


Varying  opinions  on  what  we  are  supporting.  Appears  that 
SC  interests  take  front  seat  to  what  missions  IT  systems  are 
supporting. _ 


Senior  AF  leadership  is  more  aware  of  the  need  for  the 
establishment  and  enforcement  of  enterprise  IT  policy  and 
guidance. _ 


Comparisons  easier  to  make  based  on  total  cost  of 
ownership  studies,  ROI  tracking,  etc. 


This  area  has  suffered  from  bureaucratic  paralysis  for  the 
past  several  of  years 


Development  of  comprehensive  standards,  and  their 
enforcement 


Needs  to  be  updated,  but  combination  of  IRM  strategic  plan 
produced  under  C3I  guidance  and  C&l  strat  plan  started  a 
process  that  links  to  APPG  and  POM;  efforts  now  under  way 
in  AF  CIO  to  rebuild. 


Concentrated  effort  is  making  a  difference,  but  we  need  it 
faster  and  with  money. 


Process  change  is  experiencing  somewhat  of  a  TQM 
backlash  in  the  AF  at  this  time. 


Policy  is  now  more  visible  and  defendable.  Plus  the  right 
organization  to  implement 


CIO  working  to  integrate  CCA  reguirements  into  AF 
corporate  structure  and  processes;  CIO  directed  the  first 
cross-cutting  IT  investment  review  during  the  FY03  POM 
rocess  starting  at  the  Panel  level 


Commands  build  PMs/metrics  based  on  CIO  vision. 


CIO  is  trying  to  make  significant  visibility  changes  in  this 


Evident  by  the  fact  that  IT  drives  much  of  how  we  do 
business  today. 


IT  now  an  integral  part  of  all  AF  activities  and  not  an  after¬ 
thought _ 


Believe  we  still  sometimes  put  manpower,  training,  and 
sustainment  concerns  behind  the  program  instead  of  upfront 
where  they  belong;  also  technology  keeps  outpacing  our 
ability  to  effectively  train  and  produce  current  policy. 


Believe  CIO  is  making  progress  in  getting  the  AF  to  look  big 
icture,  vice  MAJCOM/base  county  option. 


Much  more  structure  and  finding  efficiencies  when  looking  at 
IT  in  its  entirety. 


Part  2:  AF-CIO  Position  Model 


Part  2  of  the  web-based  survey  addresses  Research  Question  #2.  In  Part  2,  the 
AF-ClO  Position  Model  analysis  is  reported.  The  results  are  based  on  Bernard’s  research 
and  the  respondent’s  answers  to  questions  regarding  the  Federal  CIO  Core  Competencies 
and  the  understanding  of  the  Parsons/Thompson  theory  of  organizations.  Table  33  below 
presents  the  largest  proportional  responses  for  each  competency  and  level  of  importance 
and  organizational  level  defined  by  Parsons  and  Thompson. 


Table  33:  AF-CIO  Position  Model  Results  (Sorted  by  Importance) 


Federal  CIO  Core  Competency 

Importance 

# 

(n=66) 

Selection 

Rate 

Org  Level 

# 

(n=66) 

Selection 

Rate 

IT  Security  /  Information  Assurance 

Very 

62 

93.94% 

Technical 

30 

46.2% 

Caoital  Plannina&  Investment  Assessment 

Very 

58 

87.88% 

Institutional 

45 

69.2% 

Leadershio  /  Manaaerial 

Very 

55 

83.33% 

Manaaaiai 

37 

56.9% 

Policy  /  Oraanizational 

Vav 

55 

83.33% 

Institutional 

55 

84.6% 

IR  Strateav  &  Rannina 

Vav 

51 

77.27% 

Institutional 

42 

64.6% 

Process  /  Chanae  M  anaaement 

Vav 

45 

68.18% 

Manaaaial 

40 

62.5% 

Acquisition 

Vav 

45 

68.18% 

Manaaaial 

30 

46.2% 

Proiect  /  Proaram  Manaaement 

Vav 

39 

59.09% 

Manaaaial 

41 

63.1% 

Technical 

Vay 

35 

53.03% 

Technical 

47 

72.3% 

Somewhat 

37 

56.06% 

Technical 

43 

65.2% 

E-Gov  /  E-Bus  /  E-Comm 

Somewhat 

37 

56.06% 

Institutional 

44 

67.7% 

Performance  Assessment:  M  odel  s  and  M  ethods 

Somewhat 

33 

50.00% 

33 

50.8% 

The  results  indicate  that  as  a  large  federal  service  organization  under  the  agency 
of  the  DoD,  the  USAF  shares  some  organizational  results  similar  to  other  large  federal 
government  agencies.  There  was  no  recognizable  pattern  between  the  assigned 
importance  level  rating  of  competencies  and  the  level  at  which  they  were  identified  as 
operating  agencies.  Also,  according  to  Bernard,  key-actors  indicated  that  the  agency  CIO 
should  operate  at  a  very  high  level  and  should  not  engage  in  hands-on  management 
activities  (Bernard,  2001:  139).  Further,  sub-agency  level  CIOs  should  then  work  at  the 
level  of  implementation  and  integration,  and  focus  on  the  how-to-do-it  side  of  IRM. 
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However,  Bernard  noted  that  IT  workforce  issues,  while  seemingly  a  sub-agency 
responsibility,  are  implemented  at  the  Executive  Level  especially  in  light  of  the  imminent 
aging  workforce  and  the  possibility  of  losing  knowledge  and  skills  once  they  retire. 

A  threshold  was  established  at  45%  or  greater  for  any  particular  level  of 
importance  and  organizational  level  defined  by  the  Parsons/Thompson  model.  As 
mentioned  earlier,  Bernard  fashioned  the  Parsons/Thompson  organizational  structure 
dimension  to  his  CIO  Position  Model  in  order  to  provide  a  conceptual  model  of  how  the 
two  are  related  and  to  reveal  whether  a  basic  relationship  exists. 

The  following  table  shows  the  results  of  Part  2,  The  AF-CIO  Position  Model, 
from  the  survey.  The  highlighted  areas  under  “Level  of  Importance”  and  “Org  Level” 
show  the  highest  response  rates  for  eaeh  of  the  Federal  CIO  Core  Competencies. 
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Table  34:  AF-CIO  Position  Model  Results 


Federal  CIO  Core  Competency 

Importance 

#  Voted 

%  of 

Vote 

Org  Level 

#  Voted 

%  of 

Vote 

Policy  /  Organizational 

Very 

55 

83.3% 

Technical 

3 

4.6% 

Somewhat 

10 

15.2% 

Institutional 

55 

84.6% 

Not 

1 

1.5% 

Managerial 

7 

10.8% 

Leadership  /  Managerial 

Very 

55 

83.3% 

Technical 

4 

6.2% 

Somewhat 

9 

13.6% 

Insstitutional 

24 

36.9% 

Not 

2 

3.0% 

Managerial 

37 

56.9% 

Process  /  Change  Management 

Very 

45 

68.2% 

Technical 

8 

12.5% 

Somewhat 

20 

30.3% 

Institutional 

16 

25.0% 

Not 

1 

1.5% 

Managerial 

40 

62.5% 

IR  Strategy  &  Planning 

Very 

51 

77.3% 

Technical 

6 

9.2% 

Somewhat 

12 

18.2% 

Institutional 

42 

64.6% 

Not 

3 

4.5% 

Managerial 

17 

26.2% 

Performance  Assessment:  Models  and  Methods 

Very 

30 

45.5% 

Technical 

17 

26.2% 

Somewhat 

33 

50.0% 

Institutional 

15 

23.1% 

Not 

3 

4.5% 

Managerial 

33 

50.8% 

Project  /  Program  Management 

Very 

39 

59.1% 

Technical 

14 

21.5% 

Somewhat 

24 

36.4% 

Institutional 

10 

15.4% 

Not 

3 

4.5% 

Managerial 

41 

63.1% 

Capital  Planning  &  Investment  Assessment 

Very 

58 

87.9% 

Technical 

4 

6.2% 

Somewhat 

5 

7.6% 

Institutional 

45 

69.2% 

Not 

3 

4.5% 

Managerial 

16 

24.6% 

Acquisition 

Very 

45 

68.2% 

Technical 

8 

12.3% 

Somewhat 

20 

30.3% 

Institutional 

27 

41.5% 

Not 

1 

1.5% 

Managerial 

30 

46.2% 

E-Gov  /  E-Bus  /  E-Comm 

Very 

23 

34.8% 

Technical 

5 

7.7% 

Somewhat 

37 

56.1% 

Institutional 

44 

67.7% 

Not 

6 

16 

IT  Security  /  Information  Assurance 

Very 

62 

93.9% 

Technical 

30 

46.2% 

Somewhat 

3 

4.5% 

Institutional 

23 

35.4% 

Not 

1 

1 .5% 

Managerial 

12 

18.5% 

Technical 

Very 

35 

53.0% 

Technical 

47 

72.3% 

Somewhat 

25 

37.9% 

Institutional 

5 

7.7% 

Not 

6 

9.1% 

Managerial 

13 

20.0% 

Desktop  Technology  Tools 

Very 

21 

31.8% 

Technical 

43 

65.2% 

Somewhat 

37 

56.1% 

Institutional 

13 

19.7% 

Not 

8 

12.1% 

Managerial 

10 

15.2% 

Based  on  the  information  in  the  table  above,  conceptual  model  of  the  AF-ClO 
position  was  created. 
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Figure  9:  The  AF-CIO  Position  Model  (Adapted  from  Bernard’s  Research) 

The  consequence  of  this  model  is  the  realization  that  the  AF-ClO  operates 
primarily  in  the  Institutional  and  Managerial  Layer  of  the  USAF.  This  presumably  is  a 
fair  representation  of  exactly  where  the  USAF  desires  the  AF-CIO  to  concentrate  it’s 
time  and  resources.  As  pointed  out  earlier,  the  Technical  Level  represents  the  production 
base  or  implementation  point  of  Managerial  and  Institutional  guidance. 

Part  3:  Federal  CIO  Position  Evaluation  Method 

Part  3  of  the  web-based  survey  addresses  Research  Question  #1.  In  Part  3,  the 
enhanced  Federal  CIO  Position  Evaluation  Method  (FCPEM)  analyses  are  presented 
using  the  CIO  Role  Evaluation  Standards  developed  by  Bernard.  The  FCPEM  is 
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established  here  for  the  USAF.  The  results  of  Part  3  are  utilized  to  examine  technical  and 


substantive  compliance  with  the  CCA.  Determining  the  degree  of  compliance  in  each 
area  gives  both  a  functionally  specific,  and  in  summary,  an  overall  indication  of  support 
for  the  CIO  provisions  of  the  CCA.  Both  compliance  with  and  variance  from  CCA-CIO 
mandates  are  potentially  valuable  information  for  policy  studies  on  what  the  effect  of  this 
portion  of  the  law  has  been. 

Under  the  category  of  “Goal”,  the  selection  most  frequently  chosen  is  reflected 
for  each  Role  Evaluation  Standard.  Bernard  added  “goal  for”  and  “goal  of’  as  a  measure 
to  provide  greater  understanding  of  agency  motivations  in  implementing  the  Clinger- 
Cohen  Act  (CCA).  A  goal  for  an  agency  is  interpreted  to  be  “externally  induced”;  a  goal 
of  as  “internally  generated”.  A  “goal  for”  an  ageney  implies  the  agency  would  not  have 
aceomplished  the  role  had  it  not  been  mandated.  A  “goal  of’  means  the  agency  had 
already  been  attempting  to  implement  it  before  mandated.  In  the  case  of  the  AF-CIO 
offiee,  7  of  13  role  evaluations  standards  refleet  a  “goal  for”  the  organization.  The 
remaining  6,  ineluding  those  listed  as  “both”  are  eonsidered  “goals  of’  the  AF-CIO. 

Along  with  goals,  there  are  levels  of  eomplexity.  Bernard  added  complexity  to 
“assess  it  as  a  risk  element  and  then  mitigate  a  wider  spectrum  of  disruption  sources, 
accepting  that  this  will  not  be  a  foolproof  exereise.  Things  will  still  happen,  and  when 
they  do,  it  is  not  necessarily  the  mark  of  poor  CIO  or  agency  head  performance” 

(Bernard,  2001 :228).  The  survey  indicates  that  there  are  six  highly  complex  roles  that  the 
AF-CIO  must  perform,  some  of  them  coneurrently.  This  further  indicates  that  the  AF- 
CIO  position  is  one  that  requires  experienee  in  a  number  of  management  and  technical 


101 


areas,  as  well  as  good  organizational  and  interpersonal  skills.  The  goal  and  complexity 
results  are  listed  in  the  table  below. 

Table  35:  FCPEM  Goal  and  Complexity  Level  Results 


FCPEM  Role  Evaluation  Standards 

Goal 

Goal 

Counts 

(n=53) 

Complexity 

Complexity 

Counts 

(n=53) 

Position  Established 

For 

26 

QO  Designated  Executive  Level  IV 

Both 

19 

II— 

QOestablished  inwiling 

For 

20 

Does  document  make  IRM  principal  duty 

For 

22 

Facilitate  Reviews  for  efficient  IRM  processing 

a 

21 

High 

2 

4 

QO  supports  defining  process  need/strategies 

Both 

23 

High 

37 

Facilitate  evaluation  of  infcrrmalion  collection 

a 

19 

High 

22 

Capital  planning  process 

For 

23 

High 

36 

Architecture  to  Capital  Ranning 

For 

23 

High 

27 

ReviewIT  programfor  <10%  variance 

a 

22 

Medium 

24 

Strategic  plan  and  Performance  Report 

For 

22 

High 

24 

Workforce  Plan 

a 

20 

Medium 

25 

Annual  IRM  Progress  Reporting 

For 

31 

Medium 

24 

A  matrix  of  responses  for  the  FCPEM  Role  Evaluation  Standards  to  the  Federal 
CIO  Core  Competencies  are  included  in  Table  36  below.  Only  competencies  which  were 
selected  greater  that  30%  of  the  time  for  each  standard  is  reflected  in  the  table.  This  is  a 
subjective  call  by  the  researcher  to  present  an  overall  view  of  relevant  information. 
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Table  36:  FCPEM  Competency  Selection  Results 


FCPEM  Role  Evaluation  Standards 

Competency  (  >  30%  selection  rate) 

#  Times 

Chosen 

(n=53) 

Selection 
Rate  % 

Facilitate  Reviews  for  efficient  IRM  processing 

Policy  /  Organizational 

29 

54.72% 

'  '  1 

Leadership  /  Managerial 

27 

50.94% 

Process  Change  Management 

23 

43.40% 

i 

IR  Strategy  and  Planning 

20 

37.74% 

CIO  supports  defining  process  need/strategies 

Leadership  /  Managerial 

28 

52.83% 

IR  Strategy  and  Planning 

28 

52.83% 

Capital  Planning  &  Investment  Assessment 

27 

50.94% 

Policy  /  Organizational 

25 

47.17% 

Process  /  Change  Management 

16 

30.19% 

Facilitate  evaluation  of  information  collection 

Leadership  /  Managerial 

17 

Performance  Assessment:  Models  &  Methods 

16 

30.19% 

Policy  /  Organizational 

16 

30.19% 

Capital  planning  process 

Policy  /  Organizational 

25 

47.17% 

Leadership  /  Managerial 

25 

47.17% 

Capital  Planning  &  Investment  Assessment 

24 

45.28% 

IR  Strategy  and  Planning 

21 

39.62% 

Acquisition 

20 

37.74% 

Project/  Program  Management 

16 

30.19% 

Architecture  to  Capital  Planning 

Capital  Planning  &  InvestmentAssessment 

25 

47.17% 

Policy  /  Organizational 

22 

41.51% 

i 

Leadership  /  Managerial 

19 

35.85% 

Review  IT  program  for  <10%  variance 

Leadership  /  Managerial 

19 

35.85% 

Project/  Program  Management 

19 

35.85% 

Capital  Planning  &  InvestmentAssessment 

18 

33.96% 

Performance  Assessment:  Models  &  Methods 

18 

33.96% 

Policy  /  Organizational 

17 

32.08% 

Strategic  plan  and  Peribrmance  Report 

Leadership  /  Managerial 

25 

47.17% 

IR  Strategy  and  Planning 

25 

47.17% 

I 

Policy  /  Organizational 

23 

43.40% 

Workforce  Plan 

Leadership  /  Managerial 

27 

50.94% 

Policy  /  Organizational 

25 

47.17% 

IR  Strategy  and  Planning 

18 

33.96% 

Annual  IRM  Progress  Reporting 

Policy  /  Organizational 

31 

58.49% 

Leadership  /  Managerial 

26 

49.06% 

Performance  Assessment:  Models  &  Methods 

17 

32.08% 
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Results  of  the  FCPEM  for  the  AF-CIO 


Independent  results  to  the  13  Role  Evaluation  Standards  are  discussed  below. 

CCA/CIO  Role  #1  (Establish  a  CIO  Position/Title): 

The  USAF  has  designated  a  CIO  in  writing.  Dr.  Lawrence  Delaney,  Assistant 
Secretary  of  the  Air  Force  for  Acquisitions  and  the  Air  Force  CIO,  accomplished  this  in 
August  of  1996.  The  Current  Secretary  of  the  Air  Force,  Dr.  James  G.  Roche,  recently 
restructured  the  AF-ClO  organization  establishing  direct  reporting  between  the  Under 
Secretary  of  the  Air  Force  and  the  AF-CIO  in  November  2001. 

The  CCA/CIO  mandate  is  satisfied  since  the  USAF  has  a  CIO.  The  USAF  is  not 
a  group  of  agencies,  it  is  but  one  military  department.  Having  a  CIO  at  the  major  and 
mid-command  level  is  an  industry  and  government-proven  “best  practice”  which 
provided  additional  value  to  mission  aeeomplishment.  Given  the  IT-related  objectives  of 
Joint  Vision,  2010,  Joint  Vision  -2020,  and  the  Quadrennial  Defense  Review,  that  case 
eould  be  made  DoD-wide. 

CCA/CIO  Role  #2  (CIO  Designated  at  Executive  Level-IV): 

The  AF-CIO  is  a  member  of  the  Senior  Executive  Service.  The  AF-CIO  position 
is  designated  Executive  Level-IV  and  SES  Level  5/6  which  meets  the  intent  of  the  CCA. 

CCA/CIO  Role  #3  (CIO  reports  directly  to  the  agency  head): 

As  instrueted  by  SAF  Order  560.1,  November  26,  2001,  The  Secretary  of  the  Air 
Force,  Dr.  James  G.  Roche,  restruetured  the  AF-CIO  organization,  establishing  direct 
reporting  between  the  Under  Secretary  of  the  Air  Force  and  the  AF-CIO. 

CCA/CIO  Role  #4  (IRM  is  the  CIOs  principle  duty) 
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As  with  the  above  role,  SAF  Order  560.1,  November  26,  2001,  The  Secretary  of 
the  Air  Force,  restructured  the  AF-CIO  organization.  This  order  readdresses  IRM  as  the 
AF-CIO  primary  duty.  The  AF-ClO  is  designated  the  principal  adviser  on  information 
management,  business  processes  and  information  technology  standards.  The  AF-CIO 
and  the  Deputy  CIO  are  responsible  for  overall  implementation  of  information 
technology  management  policy  for  the  USAF. 

CCA/CIO  Role  #5  (CIO  facilitates  reviews  to  ensure  efficient  IRM  processes, 
including  reducing  information  collection  burdens  on  the  public): 

Evidence  of  compliance  is  listed  on  the  AF-CIO  web  page,  in  CIO  designation 
mission  documents,  and  in  reviews  such  as  the  EXCOM  and  CIOMB  meetings,  as  well  as 
through  the  meeting  minutes  of  the  EXCOM  meetings.  Survey  response  feedback 
indicated  that  the  AF-CIO  office  routinely  performs  this  responsibility  at  the  executive 
and  base  levels. 

CCA/CIO  Role  #6  (CIO  supports  defining  the  agency's  program  information 
needs,  strategies,  systems,  and  capabilities): 

AF-CIO  office  visibility  in  this  process  is  evident.  There  are  several  instances 
where  evidence  of  AF-CIO  activity  are  observed,  such  as  the  Information  Technology 
Management  Plan,  the  draft  IT  Strategic  Plan,  the  Air  Force  Strategic  Plan,  and  the 
program  selection  phase  of  the  capital  planning  processes.  The  Air  Force  Information 
Technology  Investment  Performance  Measurement  Guide  is  being  developed  to  support 
the  development  of  performance  measures  that  demonstrate  the  value  of  the  IT 
contribution  to  the  USFA  mission. 
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CCA/CIO  Role  #7  (CIO  heads  a  process  to  evaluate  proposed  agency  collections 
of  information): 

The  AF-CIO  office  has  a  process  to  evaluate  collection  of  information  through  the 
Air  Force  Information  Collection  and  Reports  Program.  Survey  response  feedback 
indicates  this  is  a  highly  complex  task  and  continues  to  be  a  goal  for  the  AF-ClO  office. 
References  to  the  federal  government  policy  are  made  in  the  AF-CIO  Strategic  Plan, 
indicating  acknowledgement  of  the  responsibility  for  this  role.  The  researcher  found 
evidence  of  it’s  accomplishment  through  the  USAF  report  to  the  Secretary  of  Defense, 
and  the  Annual  Report  to  Congress  and  the  President.  Efforts  are  continuing  to  establish 
best-practice  commercial  and  government  benchmarks  for  this  process. 

CCA/CIO  Role  #8  (CIO  provides  advice  to  agency  head/management  to  ensure  IT 
is  acqidred  and  IRM  done  in  accordance  with  PRA  '95  and  agency-head  priorities): 

The  AF-CIO  office  is  complying  with  the  CCA-mandated  IT  Capital  Planning 
processes.  The  AF-CIO  office  takes  part  in  or  leads  the  IT  capital  planning  and 
investment  control  process.  The  process  occurs  within  the  existing  corporate  structure 
and  the  programming  and  budget  process. 

CCA/CIO  Role  #9  (CIO  develops,  maintains,  facilitates  an  integrated  agency  IT 

architecture): 

Analysis  of  documents  and  reviews  of  the  AF-CIO  web  page  for  architecture 
initiatives  indicates  that  the  AF-CIO  office  is  leading  the  development  and 
implementation  of  integrated  IT  and  business  architectures.  The  AF-CIO  architecture 
initiatives  include  documentation  of  the  four  general  layers  (business,  data,  applications, 
and  technical)  and  a  technical  standards  reference  model.  The  AF-CIO  strategy  is  to 
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integrate  architecture  into  core  Air  Force  planning  and  system  development  processes. 
Development  of  Air  Force  architecture  products  is  being  done  within  Domain 
Architecture  Councils  led  by  operational  and  functional  communities.  The  processes 
being  used  to  develop  architecture  products  are  using  a  collaborative  model  with  cross 
Air  Force  and  joint  service  participation. 

CCA/CIO  Role  #10  (CIO  monitors/evaluates  IT  program  performance  and 
advises  continuation): 

IT-related  program  monitoring  is  being  accomplished  in  the  AF-CIO  office, 
through  the  CIO  Management  Board  (CIOMB).  The  process  occurs  within  the  existing 
corporate  structure  and  the  programming  and  budget  process.  The  AF-ClO  office 
oversees  an  integrated  review  proeess  of  IT  expenditures  AF-wide  to  support  IT  decision 
making  .  The  proeess  begins  at  the  MAJCOM  level.  The  AF-CIO  offiee  does  not  have 
broad  budget  approval  authority  over  IT  programs  AF-wide.  Briefings  given  by 
members  of  the  AF-CIO  offiee  indieate  this  impaets  CIO  effeetiveness  in  eontrolling  IT 
programs  that  are  over  budget,  or  behind  sehedule,  or  not  performing  well.  A  result  of 
this  lack  of  control  over  IT  AF-wide  programs  leads  to  suboptimizing  of  resources. 

CCA/CIO  Role  #II  (CIO  participates  in  fiscal  year  Agency  strategic  planning 
and  performance  evaluation  processes): 

The  AF-CIO  offiee  partieipates  in  high-level  agency  strategic  planning  and 
performance  evaluation  processes.  Doeument  analyses,  survey  response  feedback,  and 
AF-CIO  office  briefings  indieate  a  signifieant  involvement  in  high-level  agency  strategic 
planning.  The  goal  is  that  the  CIOs  provide  an  integrated  view  of  information  technology 
programming  and  budget  requests  and  advise  the  Air  Force  senior  leadership  to  ensure 
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that  the  funds  are  appropriately  allocated.  The  CIOs  at  the  MAJCOM  level  as  well  as 
functional  communities  are  now  working  to  establish  resource  oversight  processes  so  that 
the  AF-CIO  can  advise  USAF  leadership  on  how  to  apportion  funding  currently  allocated 
to  information  technology  to  support  Server/Network  Consolidation,  Portal  and  other  key 
IT  initiatives. 

CCA/CIO  Role  #12  (CIO  assesses  IRM  skill  requirements,  develops  strategies  to 
rectify  deficiencies  with  plans  for  hiring,  training,  and  professional  development): 

The  AF-ClO  office  is  continuing  its  assessment  of  IRM  related  skills  of  the 
present  and  future  workforce.  Evidence  suggests  that  training  and  educating  the  IT  /  IRM 
workforce  is  key  to  institutionalizing  the  principle  of  information  as  an  Air  Force 
strategic  resource  and  has  been  designated  as  being  critical  to  the  future  of  the  Air  Force. 
The  USAF  continues  to  certify  IT  and  IRM  professionals  through  the  National  Defense 
University,  Information  Resource  Management  College  (IRMC),  Fort  McNair,  Virginia. 

CCA/CIO  Role  #13.  (CIO  reports  annual  progress  in  improving  IRM  capability  to 
the  agency  head): 

The  AF-CIO  reports  annual  progress  in  improving  IRM  capability  to  the 
Secretary  of  the  Air  Force  and  the  Secretary  of  Defense.  Evidence  of  an  annual 
performance  plan  is  documented  through  the  AF-CIO  web  page.  Also,  many  public 
sources  showed  objective  evidence  of  the  progress  of  the  USAF  in  this  area.  The  plan 
submitted  is  complete  with  USAF  IT  initiatives,  progress,  goals,  measures,  and  results. 
The  plan  appears  to  be  updated  frequently.  Major  IT  initiatives  include  the  One  Air 
Force — One  Network  vision.  Air  Force  Server  Consolidation,  the  Air  Force  Portal,  the 
Global  Information  Grid,  and  AF  Way. 
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The  FCPEM  Checklist  for  the  AF-CIO 


Fashioning  the  Federal  CIO  Core  Competencies  to  the  FCPEM  evaluation  method 
provides  evaluation  criteria  for  the  entire  range  of  IRM -related  CIO  duties  described  in 
the  CCA  and  in  other  legislation  that  was  linked  to  the  CCA,  such  as  PRA'95.  The 
FCPEM  fills  a  gap  in  federal  guidance,  in  that  no  other  compliance  "checklist"  exists. 

The  following  table  provides  support  to  the  AF-ClO  office  in  regard  to  compliance  with 
the  CCA. 
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Table  37:  FCPEM  for  the  AF-CIO 


■ 

CIO  Roles,  Per  the  CCA 

The  Evaluation  Standard 

Goal 

Complexity  of 

Related  CIO 

1 

(Section  5125) 

for  Each  CIO  Role 

for/of 

Agency 

the  CIO  Role 

Area 

Com  potency 
(>30%) 

1 

Agency  establishes  a  CIO 

position/title. 

5125(a)(1)(A)&(B) 

Was  A  CIO  position 
formally  designated  and 
established? 

For 

N/A 

N/A 

OMB  96-02, 
EO-13011, 

PRA‘95 

2 

CIO  designated  at 

Executive  Level-IV 

5125  (e) 

Is  the  CIO  a  member  of  the 
Senior  Executive  Service, 
Level  IV? 

Both 

N/A 

N/A 

Title  44 

U.S.  Code 

Section  5315 

3 

CIO  reports  directly 
to  the  agency  head 

5125  (a)(1)(A)&(B) 

Is  direct  ClO-agency 
head  reporting 
established  in  writing? 

For 

N/A 

N/A 

PRA'95, 

OMB  Memo96-02 

4 

IRM  is  the  CIOs 
principle  duty. 

5125(c)(1) 

Does  the  designation 
document  make  IRM  the 
CIO's  principle  duty? 

For 

N/A 

N/A 

OMB  Memo96-02 

5 

CIO  ensures  efficient 

IRM  processes, 
including  reducing 
information  collection 
burdens  on  the  public 

5125  a(1)(c) 

Does  the  CIO  facilitate 
reviews  to  improve  IRM- 
related  processes, 
including  reducing  the 
public  information 
collection  burden? 

Of 

High 

1,  2,  3,  4 

PRA  '95, 

OMB  A-130, 

GPEA, 

GAO  Reports 

6 

CIO  supports  defining 
the  agency's  program 
information  needs, 
strategies,  systems, 
and  capabilities. 

5125  (a)(1)(c) 

Is  there  a  CIO  &  CFO 
facilitated  process  for 
identifying  all  agency 
program  IT  needs, 
strategies,  systems, 
capabilities? 

Both 

High 

2,  4,  7,  1,  3 

PRA  '95, 

OMB  A-130, 

GAO  Reports 

CIO  heads  a  process  to 
evaluate  proposed 
agency  collections  of 
information. 

5125  (a)(2) 

Does  the  CIO  facilitate 
the  evaluation  of 
information  collections 
independent  of  CIO 
program  roles? 

Of 

High 

2,  5,  1 

PRA  '95, 

8 

CIO  provides  advice  to 
agency  head/management 
to  ensure  IT  is 
acquired  &  IRM  done 
lAW  PRA  '95  and  agency 
head  priorities. 

5125  (b)(1)/5122(a) 

Does  the  CIO  facilitate 
an  IT  Capital  Planning 
process,  advise  agency 
head/mgmt,  &  ensure  IT 
is  acquired  &  IRM/ITA 
are  done  lAW  PRA'95  & 
agency  head  priorities? 

For 

High 

1,  2,  7,  4,  8,  6 

PRA  '95, 

OMB  A-130, 

OMBMemo96-02, 

FAR 

9 

CIO  develops, 
maintains,  facilitates 
an  integrated  agency 

IT  Architecture  (ITA) 

5125  (b)(2) 

Does  the  CIO  facilitate 
an  ITA  that  ties  to 

Capital  Planning  and 
follows  OMB  A-130/OMB 
97-16  format/guidance? 

For 

High 

7,  1,  2 

CIO  Council's  FEAF, 
OMB  Memo97-02, 
OMB  Memo97-16, 
OMB  A-130 

10 

CIO  monitors/evaluates 

IT  program  performance 
&  advises  continuation 

5125  (c)(2) 

Does  the  CIO  review  IT 
programs  for  <10% 
variance  in  cost, 
schedule,  performance? 

Of 

Medium 

2,  6,  7,  5,  1 

OMB  A-11, 

OMB  A-130, 

GPRA, 

PRA'95 

11 

CIO  participates  in  FY 
agency  strategic 
planning  &  performance 
evaluation  processes. 

5125  (c)(3) 

Is  there  an  agency  IT 
Strategic  Plan  and  is  it 
reflected  in  the  FY 

Strategic  Plan  and  the 
Performance  Report? 

For 

High 

2,  4,  1 

GPRA, 

OMB  A-1 1 

12 

CIO  assesses  IRM  skill 
requirements,  develops 
strategies  to  rectify 
deficiencies,  w/  plans 
for  hiring,  training, 
professional  development 
5125  (c)(3)(A),(B)&(C) 

Does  the  agency  have  a 
CIO-facilitated  IT 

Workforce  Plan  that 

addresses  needed  IRM 
skills,  training, 
hiring,  &  professional 
development? 

Of 

Medium 

2,  1,  4 

OMB  A-11, 

OMB  A-130, 

CIO  Council 

13 

CIO  reports  annual  progress 

in  improving  IRM  capability 
to  agency  head.  5125 
(c)(3)(D) 

Does  the  CIO  report  in 
writing  to  the  agency  head 
each  year  on  how  1 RM 
capability  is  improving? 

For 

Medium 

1, 2,  5 

OMB  A-11, 

PRA'95 

V.  Discussion 


Overview 

This  chapter  discusses  the  results  obtained  in  chapter  four  along  with 
implications,  limitations  of  the  study,  and  suggestions  for  future  research.  This  chapter 
offers  conclusions  to  the  research  by  summarizing  the  research  done  in  the  previous 
chapters.  This  study  relied  on  a  self-report  of  impacts  of  the  Clinger-Cohen  Act  on  the 
USAF.  Results  of  the  content  analysis  of  the  respondent  perceptions  are  reported.  These 
data  are  used  to  answer  the  investigative  questions,  as  well  as  make  recommendations 
regarding  future  implications  of  the  CCA  and  its  impact  on  the  USAF. 

Discussion  of  Research  Questions 

Research  Question  #1) 

Are  the  USAF  implementation  efforts  consistent  with  the  intent  of  the 
CCA? 

This  study  shows  that  the  USAF  and  the  AF-CIO  Office  is  highly  compliant  with 
the  requirements  of  the  CCA  in  all  areas  evaluated  using  the  FCPEM.  The  USAF  has 
set-up  its  CIO  position  in  accordance  with  the  mandates  of  the  Clinger-Cohen  Act.  The 
FCPEM  was  specifically  designed  to  evaluate  compliance  with  mandated  CIO  roles 
against  a  matrixed  set  of  Role  Evaluation  Standards  and  the  Federal  CIO  Core 
Competencies. 
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Research  Question  #2) 

To  what  extent  can  a  model  of  the  AF-CIO  position  illustrate  the  level  of 
the  AF-CIO  office  implementation  efforts? 

This  study  reports  the  results  of  the  Federal  CIO  Position  Model  representing  the 
AF-CIO.  Bernard’s  Federal  CIO  Position  Model  was  usable  for  this  task.  The  model 
was  tailored  for  the  AF-CIO  position  by  recommendation  from  Bernard  (Bernard, 

2001b).  The  respondents  responded  to  the  model  in  a  meaningful  way.  The  respondents 
were  able  to  respond  to  the  questions  from  Bernard’s  FCPEM  model,  indicating  it  was 
appropriate  for  the  AF-CIO  organization.  The  results  of  the  analyses  made  sense  in  light 
of  the  purpose  of  the  study.  This  model  appears  to  be  appropriate  because  it  recognizes 
the  social/managerial  characteristics  of  the  CIO  position  and  the  organizational 
environment  in  which  the  CIO  works.  Bernard’s  validation  of  the  model  made  it 
applicable  to  this  study.  The  model  depicts  what  a  CIO  should  know  and  where  that 
knowledge  should  be  applied  in  the  organization.  It  also  provides  a  greater  understanding 
of  how  the  AF-CIO  position  relates  to  IRM  policy  implementation  and  how  to  evaluate  it 
in  terms  of  the  CCA 

The  model  is  representative  of  current  events  in  the  USAF.  Recently,  in  a  letter  to 
Deputy  Chief  of  Staff  (DCS)  for  Communications  and  Information  (AF/SC)  dated  25  Jan 
02,  the  AF-CIO  re-delegated  a  large  amount  of  its  inherent  technical  responsibilities  to 
AF/SC  in  preparation  for  transition  to  a  new  organizational  structure  and  creation  of  the 
office  of  the  DCS  for  Warfighting  Integration  (AF/XI),  and  the  Directorate  of 
Communications  and  Operations  (AF/ILC).  AF/XI  and  AF/ILC  subsumes  AF/SC  by  1 
April  02  (DAF,  2002a;  DAF,  2002b). 
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The  organizational  structure  change  supports  the  results  of  this  study.  Figure  19 
in  Chapter  4  depicts  the  AF-CIO  Position  Model  that  relates  to  this  recent  change  in 
organizational  structure.  The  organizational  structure  change  positions  the  AF-ClO  to 
provide  enterprise  architecture,  framework  policy,  and  business  process  reengineering 
guidance  for  the  USAF.  This  is  characteristic  of  the  managerial  and  institutional  level  of 
an  organization  as  identified  in  the  Parsons/Thompson  model.  AF/XI  and  AF/ILC  will  be 
responsible  to  implement  AF-CIO  guidance  across  the  USAF.  This  is  also  characteristic 
of  the  Parsons/Thompson  model  in  that  the  core  technical  processes,  the  product  and 
service  producing  division  of  the  organization,  are  responsible  to  implement  direction 
from  the  institutional  and  managerial  levels. 

Research  Question  #3) 

What  outcomes  can  be  found  as  a  result  of  the  CCA  and  the  AF-CIO 
office. 

A)  The  Federal  CIO  Core  Competencies. 

48  outcome/impacts  areas  were  reported.  Over  95%  of  the  comments  received 
were  categorized  as  positive.  One  of  the  most  frequently  reported  positive  responses 
regarded  a  standardized  approach  to  accomplishing  CIO  duties.  For  example,  standard 
approaches  to  IT  services  and  performance  measures  have  reportedly  made  an  impact  on 
IRM  processes  and  implementation  efforts.  Another  example  relates  to  a  teaming 
approach  that  optimizes  base  resources.  Localized  working  groups  have  formed  that 
capitalize  on  information  sharing,  policy  implementation,  and  IRM. 
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Less  than  5%  of  the  comments  received  were  classified  as  negative  outcomes.  An 
example  of  one  negative  comment  stated  that  the  USAF  has  too  many  System  Program 
Offices  running  high  overheads,  yet  share  little  information  with  the  rest  of  the  USAF. 

B)  The  performance  based  aspects  of  the  CCA.  Section  5123. 

65  outcome/impact  areas  were  reported.  Over  75%  of  the  comments  received 
were  categorized  as  positive.  The  most  frequently  reported  positive  response  related  to 
the  USAF  incorporation  of  business  or  industry  models  when  developing  IT/IRM  policy, 
managing  projects,  and  establishing  IT  performance  measures.  While  there  were  a  few 
comments  that  suggested  caution  with  this  approach  due  to  the  nature  of  military 
purposes,  they  generally  supported  benchmarking  from  industry.  Another  example 
regards  the  results  of  what  is  commonly  referred  to  as  the  “famous  west  coast  meetings” 
by  USAF  senior  leaders.  The  USAF  and  AF-CIO  IT  initiatives  that  resulted  from  the 
meeting  were  generally  regarded  as  positively  influencing  the  direction  of  IT 
performance  in  the  future.  However,  a  few  comments  mentioned  that  comparing  the 
USAF  to  Silicon  Valley  and  their  centralization  approach  was  irrational. 

Less  than  25%  of  the  comments  received  were  classified  as  negative.  An  example 
of  an  area  of  negative  comments  related  to  the  USAF  IT  budget.  Apparently  some 
organizations  have  lost  significant  amounts  of  their  budgets  because  they  could  not 
demonstrate  improvements  had  occurred  in  their  operational  missions  as  a  result  of  a 
previously  implemented  IT  investment.  Another  example  of  a  high  number  of  negative 
responses  simply  stated  that  it  was  too  soon  to  tell  what  the  impacts  are  in  the  area  of 
performance  and  results  based  measurements.  The  researcher  categorized  this  as  a 
negative  comments  because  of  the  length  of  time  the  mandates  have  been  in  affect. 
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Although  it  was  apparent  from  the  responses  that  enforcing  the  requirements  has  been 
difficult  for  a  number  of  political  and  economic  reasons. 

C)  Key  IRM  achievement  areas  from  the  public  and  the  private  sector. 

63  outcome/impact  areas  were  reported.  Nearly  60%  of  the  comments  received 

were  categorized  as  negative.  The  section  asks  “Has  the  AF-CIO  Office  achieved 
significant  outcomes  related  to  [Key  IRM  Achievement  Areas  identified  through  previous 
research  and  reports].”  The  most  frequent  negative  response  reported  was  that  while  the 
outcome/impacts  may  be  realistic  to  IT/IRM,  it  was  not  realistic  to  USAF  operations. 
Reasons  behind  this  response  range  from  explanations  that  “it’s  too  early  to  make  an 
assessment,”  to  citing  the  USAF  is  going  the  right  direction  but  “lacks  funding,  staff,  and 
leadership  at  the  eurrent  time.”  Other  negative  responses  mentioned  much  of  the  IT 
policies  and  vision  of  the  AF-CIO  offiee  is  not  understood,  citing  that  carryovers  from 
the  Total  Quality  Management  (TQM)  eulture  hinders  understanding,  and  reporting. 

The  positive  outeomes/impaets  in  this  section  relate  to  the  effort  the  AF-CIO 
office  has  made  in  leading  IRM  transformation  in  the  USAF.  Many  respondents 
recognized  that  it  may  be  too  early  to  compare  measurements  of  Key  IRM  Achievement 
Areas  in  the  private  sector  to  the  USAF.  They  also  mentioned  that  the  AF-CIO  office  is 
proceeding  in  the  right  direction,  evidenced  by  an  enterprise  approach,  network-centric 
operations  support,  and  a  responsive  organizational  structure  to  carryout  strategic  plans. 

D)  The  roles  and  responsibilities  of  the  AF-CIO. 

In  January  of  2002,  the  AF-CIO  office  outlined  the  roles  and  responsibilities  of 
MAJCOM  CIOs  and  HAF  Functional  CIO  representatives.  The  descriptions  are  found  in 
Appendix  C.  Survey  respondents  were  asked  to  judge  this  product.  85%  felt  it  was 
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appropriate.  80%  felt  it  should  not  be  changed  at  all.  When  asked  about  roles  in  terms  of 
the  level  they  are  currently  serving,  more  than  half  stated  they  performed  all  of  the  roles 
and  responsibilities  cited  by  the  AF-CIO  office. 

The  analysis  of  shortfalls  revealed  Manpower  and  Staffing  being  cited  as  the 
largest  shortfall  by  nearly  50%  of  the  respondents.  Funding  was  a  close  second  with  a 
selection  rate  of  over  33%.  An  implication  of  this  data  is  that  while  leadership  is  dealing 
with  the  burden  to  make  major  changes  related  to  CIO  duties,  their  jobs  are  made  even 
more  difficult  by  the  fact  that  they  lack  the  resources  of  manpower  and  funding  to  carry 
them  out.  Many  respondents  offered  suggestions.  For  example,  some  respondents 
related  that  there  is  no  way  to  get  visibility  into  what  functional  users  are  spending  on  IT. 
Further,  they  “need  this  type  of  data  to  be  able  to  show  how  consolidated  efforts  can  save 
the  AF  and  MAJCOMs  funds,  manpower,  etc.”  Others  explain  that  with  greater 
responsibility  for  capital  planning  and  investment  control  in  the  AF-ClO  hands,  “...it 
somehow  gets  lost  at  the  MAJCOMs  when  the  money  for  IT  seems  to  find  its  way  into 
budgets  other  than  the  SC's.”  Lastly,  suggestions  were  made  regarding  the  participation 
of  the  AF-CIO  in  the  command/functional  coordination  with  Pentagon  Functional 
organizations  where  “lots  of  money  is  being  wasted  looking  strictly  at  functional 
requirements  only,  without  regard  to  organic  support  at  the  local  bases”,  suggesting 
optimization  of  resources  could  result  if  the  suggestions  are  followed. 
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Research  Question  #4) 

Have  changes  in  USAF  operations  occurred  as  a  result  of  instituting  an 
AF-CIO  office? 

Based  on  the  self  report  responses  from  Part  1  of  the  web-based  survey,  and  a 
content  analysis  of  those  responses  by  the  researcher,  the  evidence  supports  the 
conclusion  that  many  changes  have  occurred  in  the  USAF  as  a  result  of  the  CCA  and  the 
AF-ClO  office.  Over  20  areas  were  identified  by  the  researcher  as  having  notable 
changes,  based  on  the  content  analysis  of  the  perception  of  the  respondents.  While  a 
number  of  the  impact  areas  cited  positive  changes,  others  revealed  that  possible 
confusion  and  a  lack  of  action  have  effected  little  change. 

Though  it  has  been  nearly  six  years  sinee  the  passage  of  the  CCA,  it  was  noted 
that  a  large  number  of  respondents  pereeived  it  was  generally  to  soon  to  tell  if  the  CCA 
has  made  ehanges  in  the  USAF.  This  eould  possibly  be  explained  by  the  amount  of 
ehange  that  has  been  oeeurring  at  the  highest  levels  of  the  AF-CIO  office,  such  as 
changes  of  leadership  and  organizational  strueture  oeeurring  continuously  since  1998. 
Lending  to  this  observation  is  the  pereeption  that  shortages  in  staffing  and  funding  make 
it  diffieult  to  prioritize  and  exeeute  mission  and  support  operations. 

What  is  commonly  known  as  the  USAF  “IT  Initiatives”  garnered  a  large  number 
of  positive  comments.  IT  Initiatives  ineluded  projeets  such  as  AFWay,  the  AF  Portal, 
AF  E-mail/Server/Network  Consolidation,  and  Improved  Visibility  in  IT  Expenditures. 
The  IT  Initiatives  resulted  from  the  IT  Summit  after  a  visit  to  successful  IT  industry 
performers  by  top  Air  Force  offieials.  Related  to  this  change  was  an  equally  large 
number  of  comments  related  to  Business  Case/Model  Development.  Several  comments 
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noted  a  change  in  focus  to  IRM  as  business  entities,  as  opposed  to  IT  being  a  utility 
provided  by  a  support  organization. 

Research  Question  #5) 

How  have  these  changes  impacted  the  USAF? 

Based  on  the  self  reported  responses  from  Part  1  of  the  web-based  survey,  and  a 
content  analysis  of  those  responses  by  the  researcher,  the  data  suggests  there  are 
perceived  changes  in  the  USAF  as  a  results  of  the  CCA  and  that  these  changes  have 
purportedly  impacted  the  USAF  in  a  number  of  ways.  The  study  suggests  that,  overall 
the  changes  mentioned  previously  have  impacted  the  USAF.  A  thematic  summary 
technique  was  used  to  categorize  the  changes.  Categories  of  comments  were  judged  on 
themes  rather  than  exaet  wording  or  word  eounts  making  it  flexible  enough  to  combine 
like  themes  for  the  purpose  of  summarizing  how  the  perceived  changes  have  impacted  to 
the  USAF  sinee  the  CCA.  It  eannot  be  said  eonelusively  that  these  changes  have 
occurred,  only  that  an  observation  of  the  respondents  perceptions  suggest  these  changes 
have  impacted  the  USAF  as  a  result  of  the  CCA  and  the  influence  of  the  AF-CIO  office. 

1.  IRM  responsiveness  eontributes  to  effective  mission  accomplishment. 

2.  Strategic  planning  includes  IT  and  the  information. 

3.  Technology  has  improved  effieieney. 

4.  Baseline  performance  measurements  are  more  realistic. 

5.  Alignment  of  organizational  structure  provides  critical  services. 

6.  Standardization  of  policy  and  processes  optimizes  IT  resources. 

As  evidenced  in  previous  chapters,  research  into  the  management  of  information 
and  information  resources  has  focused  on  these  particular  areas  over  the  past  few  years. 
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Limitations 


As  with  all  studies,  there  are  limitations.  This  study  relied  on  a  self-report  of 
impacts  of  the  Clinger-Cohen  Act  on  the  USAF.  Therefore,  this  study  can  be  no  more 
accurate  than  the  respondent’s  knowledge,  experience  of,  and  willingness  to  disclose 
their  perceptions  of  what  they  believe  the  CCA  impacts  to  be  on  the  USAF.  The  survey 
used  in  this  study  posed  the  question  in  the  form  “what  changes  have  occurred  as  a  result 
of  the  CCA. . .”  in  specific  areas  of  interest  to  the  researcher.  However,  it  did  not 
specifically  address  other  areas  where  the  CCA  may  have  had  impacted.  A  follow  on 
survey  that  addresses  this  may  produce  different  results. 

The  survey  instrument  administered  in  this  study  was  based,  in  part,  on  a 
validated  instrument  that  was  originally  administered  though  face-to-face  interviews. 
Though  it  was  based  on  research  undertaken  by  others,  it  had  not  been  previously 
validated  as  a  web  based  survey  instrument.  Further  validation  of  the  survey  should  gain 
more  accurate  and  generalizable  results. 

Some  of  the  stratification  information  was  lost  during  the  process  of  data 
collection.  However,  statistical  analysis  of  the  data  suggests  that  this  did  not  appreciably 
alter  the  analysis  or  conclusions. 

The  focus  of  this  study  was  on  USAF  issues  and  AF  CIO  leaders/managers,  not 
others  with  potentially  useful  information.  The  study  proposed  that  asking  IRM  leaders 
at  various  organizational  levels  in  the  USAF  would  likely  be  the  best  approach  to 
determining  what  impacts  the  CCA  has  had  on  the  USAF.  A  follow-on  study  including 
individuals  in  the  enlisted  corps  or  various  operations/support  organizations  may  produce 
different  results. 
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While  the  goal  was  to  survey  the  entire  population  of  AF  CIO  leaders/managers, 
about  20%  were  not  reached.  The  AFIT/RR  database  used  for  identifying  the  theoretical 
population  appeared  to  have  correct  individual  personnel  data  in  the  database,  but 
incorrect  or  non-updated  data  fields  specific  to  the  duty  location  where  the  individuals  are 
actually  assigned.  Losing  partial  capability  to  conduct  a  census  may  have  impacted  the 
reliability  and  accuracy  of  the  results.  Also,  it  is  impossible  to  know  exactly  how  many 
participants  attempted  to  complete  the  survey  but  were  unable  due  to  technical  difficulties 
such  as  web  server  errors  or  communications  failures,  but  few  people  actually  reported 
such  problems  to  the  researcher. 

Finally,  the  hypothesis  test  at  an  alpha  level  of  .05  resulted  in  2  of  the  42 
questions  asked  being  rejected.  At  the  95%  level,  it  is  statistically  likely  that  this  would 
happen  through  sampling  error.  As  a  result,  it  is  uncertain  whether  the  categories  of 
Project  and  Program  Management,  and  Budget  Requests  may  be  a  statistical  artifact.  A 
follow-on  study  may  produce  different  results. 

Recommendations  for  Future  Research 

This  study  applied  a  model  to  assess  compliance  with  the  Clinger  Cohen  Act 
(CCA)  by  the  USAF  and  the  impacts  it  has  had  as  a  result.  Using  a  model  to  verify 
compliance  with  federally  mandated  legislation  is  just  one  approach.  Other  approaches 
could  be  pursued  to  research  the  impacts.  A  verification  of  the  results  of  this  exploratory 
study  through  other  methodologies  could  be  undertaken.  Verifying  the  results  of  this 
study  would  confirm  if  the  two  of  forty  questions  where  equality  between  the  populations 
was  rejected  is  repeatable,  or  if  they  exist  as  statistical  artifacts.  Conducting  research  on 
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specific  topics  such  as  financial  assessment,  interoperability  assessment,  strategic  IT 
assessment,  or  IT  performance  assessment  could  further  the  research  and  add  to  the  body 
of  knowledge  of  IRM  research. 

Another  possible  research  effort  could  be  conducted  to  compare  the  branches  of 
the  military  service.  Undertaking  similar  studies  of  other  military  services  for 
comparison  could  provide  a  plausible  list  of  DoD  best  practices  or  areas  to  avoid.  The 
results  could  benefit  IRM  researchers  as  well  as  practitioners. 

Researchers  could  conduct  studies  over  time  to  follow  the  continued  evolution  of 
impacts  of  Clinger-Cohen  Act.  Academics  could  replicate  or  otherwise  explore  the 
hypothesis  tests  conducted  in  this  study  to  test  for  validity  of  research  findings.  One 
approach  would  be  to  take  the  findings  of  this  research  and  develop  hypotheses  and  a 
researeh  design  to  quantify  the  outcomes  in  terms  of  correlation  and  statistical  power. 

Lastly,  using  the  results  of  this  study,  a  follow-on  study  could  be  undertaken  to 
explore  how  the  AF-CIO  has  addressed  the  IT  spending  portion  of  the  Clinger-Cohen 
Act.  The  study  explore  the  AF-CIO  office  in  its  efforts  to  reduced  overall  IT 
expenditures,  increase  the  performance  of  IT  programs,  or  improve  the  quality  and 
impact  of  federal  IT  management. 

Conclusions 

Responses  to  the  web-based  survey  substantiate  the  USAF  is  in  compliance  with 
the  CCA  according  to  the  analysis  of  survey  respondents.  The  Clinger-Cohen  Act 
requires  that  agencies  appoint  Chief  Information  Officers  and  implement  procedures  to 
improve  capital  planning,  performance  measurement  and  enterprise  architectures  to 
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ensure  efficient  management  of  IT.  Bernard’s  FCPEM  provides  a  means  to  determine 
compliance  with  the  CCA.  This  research  concludes  the  USAF  is  highly  compliant  with 
the  CCA. 

Results  from  this  study  suggest  that  the  Clinger-Cohen  Act  impacts  have  had  a 
positive  affect  on  the  USAF  in  key  areas.  Key  areas  have  been  identified  by  the  survey 
respondents  where  results  have  been  experienced  and  changes  have  been  made.  Several 
respondents  stated  it  is  too  early  to  tell  what  impacts  have  been  made  yet.  Even  today 
these  areas  are  being  closely  studied  for  their  potential  to  affect  changes  in  the  USAF. 

Evidence  suggests  that  Bernard’s  model  is  relevant  to  AF-CIO  position.  This 
research  demonstrated  the  first  use  of  the  FCPEM  and  CIO  Position  Model  for  CCA 
compliance  and  an  explanation  from  an  organizational  theory  perspective  for  a  military 
serviee  under  the  Department  of  Defense  (DoD).  The  model  depicts  what  a  CIO  should 
know  and  where  that  knowledge  is  to  be  applied  in  the  organization. 

Responses  reveal  impaet  areas  for  further  study.  The  continued  and  future  trends 
and  developments  that  ean  be  expeeted  by  the  AF-ClO,  and  those  involved  with  the 
management  of  information,  are  likely  to  eover  three  broad  areas;  1)  involvement  in 
high-level  strategic  business  planning,  2)  foeus  on  technology  as  business  opportunities 
and  solutions,  and  3)  organizational  culture  and  structure  in  the  provisioning  of  critical 
services. 

This  study  was  conducted  to  explore  and  better  understand  the  impacts  of  the 
CCA  on  the  USAF.  It  is  intended  the  information  gathered  could  be  useful  to  the  AF- 
CIO  office,  as  well  as  information  researchers.  As  the  Air  Force's  deployment  and  use  of 
information  initiatives  evolves,  the  needs  of  its  members  will  change.  This  exploratory 
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study  of  the  impacts  of  the  Clinger-Cohen  Act  on  the  USAF  provides  an  interim  step  in 
the  assessment  of  IT  and  IRM  within  the  Air  Force  organization. 
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Appendix  A 


Clinger-Cohen  Act  of  1996 

Reporting:  SEC.  5123.  PERFORMANCE  AND  RESULTS-BASED  MANAGEMENT. 

In  fulfilling  the  responsibilities  under  section  3506(h)  of  title 
44,  United  States  Code,  the  head  of  an  executive  agency  shall— 

(1)  establish  goals  for  improving  the  efficiency  and 
effectiveness  of  agency  operations  and,  as  appropriate,  the 
delivery  of  services  to  the  public  through  the  effective  use  of 
information  technology; 

(2)  prepare  an  annual  report,  to  be  included  in  the  executive 
agency's  budget  submission  to  Congress,  on  the  progress  in 
achieving  the  goals; 

(3)  ensure  that  performance  measurements  are  prescribed  for 
information  technology  used  by  or  to  be  acquired  for,  the 
exeeutive  ageney  and  that  the  performanee  measurements  measure 
how  well  the  information  teehnology  supports  programs  of  the 
exeeutive  ageney; 

(4)  where  eomparable  proeesses  and  organizations  in  the 
publie  or  private  seetors  exist,  quantitatively  benchmark 
agency  process  performanee  against  sueh  proeesses  in  terms  of 
eost,  speed,  produetivity,  and  quality  of  outputs  and  outcomes; 

(5)  analyze  the  missions  of  the  exeeutive  ageney  and,  based 
on  the  analysis,  revise  the  exeeutive  ageney's  mission-related 
processes  and  administrative  proeesses  as  appropriate  before 
making  signifieant  investments  in  information  technology  that 
is  to  be  used  in  support  of  the  performanee  of  those  missions; 
and 

(6)  ensure  that  the  information  security  policies, 
procedures,  and  practices  of  the  executive  agency  are  adequate. 

(c)  DUTIES  AND  QUALIFICATIONS-  The  Chief  Information  Officer  of 
an  agency  that  is  listed  in  section  901(b)  of  title  31,  United 
States  Code,  shall— 

(1)  have  information  resources  management  duties  as  that 
official's  primary  duty; 

(2)  monitor  the  performance  of  information  technology 
programs  of  the  agency,  evaluate  the  performance  of  those 
programs  on  the  basis  of  the  applicable  performance 
measurements,  and  advise  the  head  of  the  agency  regarding 
whether  to  continue,  modify,  or  terminate  a  program  or  project; 
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and 

(3)  annually,  as  part  of  the  strategic  planning  and 
performance  evaluation  process  required  (subject  to  section 
1 1 17  of  title  31,  United  States  Code)  under  section  306  of 
title  5,  United  States  Code,  and  sections  1 105(a)(29),  1115, 

1116,  1117,  and  9703  of  title  31,  United  States  Code-- 

(A)  assess  the  requirements  established  for  agency 
personnel  regarding  knowledge  and  skill  in  information 
resources  management  and  the  adequacy  of  such  requirements 
for  facilitating  the  achievement  of  the  performance  goals 
established  for  information  resources  management; 

(B)  assess  the  extent  to  which  the  positions  and 
personnel  at  the  executive  level  of  the  agency  and  the 
positions  and  personnel  at  management  level  of  the  agency 
below  the  executive  level  meet  those  requirements; 

(C)  in  order  to  rectify  any  deficiency  in  meeting  those 
requirements,  develop  strategies  and  specific  plans  for 
hiring,  training,  and  professional  development;  and 

(D)  report  to  the  head  of  the  agency  on  the  progress 
made  in  improving  information  resources  management 
capability. 

SEC.  5303.  REPORT. 

(a)  REQUIREMENT-  Not  later  than  180  days  after  the  completion  of 
a  pilot  program  under  this  title,  the  Administrator  shall— 

(1)  submit  to  the  Director  a  report  on  the  results  and 
findings  under  the  program;  and 

(2)  provide  a  copy  of  the  report  to  Congress. 

(b)  CONTENT-  The  report  shall  include  the  following: 

(1)  A  detailed  description  of  the  results  of  the  program,  as 
measured  by  the  criteria  established  for  the  program. 

(2)  A  discussion  of  any  legislation  that  the  Administrator 
recommends,  or  changes  in  regulations  that  the  Administrator 
considers  necessary,  in  order  to  improve  overall  information 
resources  management  within  the  federal  government. 

(3)  COMPTROLLER  GENERAL  REVIEW  AND  REPORT-  (A)  Not  later  than 
three  years  after  the  date  on  which  the  pilot  program  is 
established,  the  Comptroller  General  of  the  United  States  shall 
review  the  pilot  program  and  report  to  the  Congress  on  the 
results  of  the  pilot  program. 

(B)  The  report  shall  include  the  following: 

(i)  An  evaluation  of  the  extent  to  which  there  is 
competition  for  the  orders  placed  under  the  pilot  program. 

(ii)  The  effect  that  the  streamlined  procedures  under  the 
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pilot  program  have  on  prices  charged  under  multiple  award 
schedule  contracts. 

(iii)  The  effect  that  such  procedures  have  on  paperwork 
requirements  for  multiple  award  schedule  contracts  and 
orders. 

(iv)  The  impact  of  the  pilot  program  on  small  businesses 

and  socially  and  economically  disadvantaged  small  businesses. 
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Appendix  B 


Roles,  responsibilities,  and  accountability.  CCA  section  5125  ; 

(b)  GENERAL  RESPONSIBILITIES  -  The  Chief  Information  Officer  of  an 
executive  agency  shall  be  responsible  for— 

(1)  providing  advice  and  other  assistance  to  the  head  of  the  executive 
agency  and  other  senior  management  personnel  of  the  executive  agency  to  ensure 
that  information  technology  is  acquired  and  information  resources  are  managed 
for  the  executive  agency  in  a  manner  that  implements  the  policies  and  procedures 
of  this  division,  consistent  with  chapter  35  of  title  44,  United  States  Code,  and  the 
priorities  established  by  the  head  of  the  executive  agency; 

(2)  developing,  maintaining,  and  facilitating  the  implementation  of  a 
sound  and  integrated  information  technology  architecture  for  the  executive 
agency;  and 

(3)  promoting  the  effective  and  efficient  design  and  operation  of  all  major 
information  resources  management  processes  for  the  executive  agency,  including 
improvements  to  work  processes  of  the  executive  agency. 

(c)  DUTIES  AND  QUALIFICATIONS-  The  Chief  Information  Officer  of  an 
agency  that  is  listed  in  section  901(b)  of  title  31,  United  States  Code,  shall— 

(1)  have  information  resources  management  duties  as  that  official's 
primary  duty; 

(2)  monitor  the  performance  of  information  technology  programs  of  the 
agency,  evaluate  the  performance  of  those  programs  on  the  basis  of  the  applicable 
performance  measurements,  and  advise  the  head  of  the  agency  regarding  whether 
to  continue,  modify,  or  terminate  a  program  or  project;  and 

(3)  annually,  as  part  of  the  strategic  planning  and  performance  evaluation 
process  required  (subject  to  section  1 1 1 7  of  title  31,  United  States  Code)  under 
section  306  of  title  5,  United  States  Code,  and  sections  1 105(a)(29),  1115,  1116, 
1117,  and  9703  of  title  31,  United  States  Code— 

(A)  assess  the  requirements  established  for  agency  personnel 
regarding  knowledge  and  skill  in  information  resources  management  and  the 
adequacy  of  such  requirements  for  facilitating  the  achievement  of  the 
performance  goals  established  for  information  resources  management; 

(B)  assess  the  extent  to  which  the  positions  and  personnel  at  the 
executive  level  of  the  agency  and  the  positions  and  personnel  at  management 
level  of  the  agency  below  the  executive  level  meet  those  requirements; 

(C)  in  order  to  rectify  any  deficiency  in  meeting  those 
requirements,  develop  strategies  and  specific  plans  for  hiring,  training,  and 
professional  development;  and 

(D)  report  to  the  head  of  the  agency  on  the  progress  made  in 
improving  information  resources  management  capability. 
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Appendix  C 


MAJCOM  CIOs  and  HAF  Functional  CIO  Representatives 
Roles  and  Responsibilities 
2  January  2002 

The  Air  Force  has  been  remarkably  successful  in  exploiting  information 
technology  to  become  the  most  powerful  force  on  earth.  Now,  our  leaders  expeet  even 
more  from  technology— and  we  can  deliver.  This  paper  outlines  our  roles  and 
responsibilities  for  working  together,  as  a  CIO  Community,  so  we  ean  do  even  more  than 
we  have  done  in  the  past.  As  CIOs,  our  primary  duty  is  to  advise  the  mission  leaders  on 
the  most  effective  use  of  IT.  As  a  community,  our  common  obligation  is  to  share  ideas, 
insights,  and  inspirations  to  help  us  collectively  meet  or  exeeed  mission  objeetives. 
Whether  implementing  the  global  strike  task  force  or  taking  better  eare  of  Air  Foree 
people,  senior  leadership  is  counting  on  IT  and  the  CIO  Community  to  make  a  differenee. 

The  following  roles  and  responsibilities  provide  a  top-level  framework  to  earry 
out  our  challenge.  Sound  capital  planning  and  investment  ean  give  senior  leadership  a 
decision  support  structure  for  IT  investments.  Practical  acquisition  principles  ean  ensure 
reliable,  consistent  development  and  fielding  of  computer  systems  (hardware,  software, 
operational  processes,  etc.).  Performance  measures  are  neeessary  to  set  our  goals  and 
demonstrate  our  progress  toward  achieving  them.  Information  assuranee  must  be 
“designed- in”  and  an  integral  part  of  all  our  processes  and  deeisions.  We  must  be  eareful 
to  distinguish  between  standards  (which  we  want)  and  standardization  (whieh  we  may  not 
want  or  may  want  in  specific  instances).  Strategic  planning  is  the  handshake  (on  mutual 
expectations  of  mission  improvement)  among  the  commanders,  functionals,  and  the  IT 
community.  Architectures  must  be  developed  and  used  to  drive  investments  and 
capabilities  fielded.  Training  and  education  gives  the  workforce  the  IT  skills,  knowledge, 
and  ability  to  carry  out  their  duties.  Finally,  knowledge  management  provides  the 
foundation  to  tap  into  the  workforce  mission/business  intelligence  and  convert  that 
expertise  into  electronic  intellectual  capital. 

Capital  Planning  and  Investment  Control: 

Establish  Capital  Planning  and  Investment  Control  processes  to  oversee 
management  and  evaluation  of  MAJCOM/Functional  IT  investments  (based  on  sound 
business  process  analysis/reengineering  and  business  case  analysis).  Identify  IT  contract 
efforts  which  experience  significant  deviation  Ifom  cost,  performance,  or  schedule  goals. 

Integrate  IT  resource  decisions  with  the  HAF  and  MAJCOM  planning,  budgeting, 
and  program  management  processes  and  priorities. 

Analyze  Functional  or  MAJCOM  budget  inputs  and  expenditures  to  certify  IT 
budget  requests  are  clearly  identified  and  comply  with  Air  Force  IT  guidelines  regarding 
standards,  architectures,  and  business  process  improvements. 
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Determine  whether  the  functional  mission  or  MAJCOM  function  to  be  supported 
by  new  IT  investments  should  be  performed  by  AF  personnel,  contracted  support,  or 
privatized. 

Ensure  appropriate  reviews  of  the  Air  Force  systems  compliance  databases  to 
optimize  reuse  and  minimize  duplication  of  systems/applications.  Include  proposed  new 
systems  in  the  systems  compliance  database.  Where  appropriate,  use  a  modular 
contracting  approach  (as  defined  by  Clinger-Cohen  Act)  which  may  be  helpful  in 
evaluating  IT  contracting  effectiveness. 

Information  Technology  Acquisition: 

Advise  and  assist  the  Fleadquarters  Air  Force  Functional  two-letter,  or 
MAJCOM/CC  and  other  senior  MAJCOM  management  to  acquire  IT  in  accordance  with 
enterprise  objectives.  Life  cycle  management  should  focus  on  projected  versus  actual 
costs,  benefits  and  risks,  lAW  AF-CIO  policies  and  procedures. 

Ensure  a  Command,  Control,  Communications,  Computers  and  Intelligence 
Support  Plan  (C41SP),  Certificates  of  Networthiness,  and  Certificates  to  Operate  are 
obtained  for  all  new  and  major  modifications  to  existing  functional  systems. 

Participate  in  Clinger-Cohen  Act  Compliance  reviews  for  AF  systems  and 
conduct  reviews  as  required  for  MAJCOM  or  Functional  unique  systems. 

Ensure  IT  acquired  meets  the  requirements  of  5  U.S.C  Section  508,  or  can  be 
adapted  to  meet  See  508,  to  ensure  equal  aeeess  to  information  environments  for  people 
with  disabilities. 

Performance  Measures: 

Assist  HAF  Funetional  areas/  MAJCOM  level  and  Functional  organizations  in 
establishing  goals  for  improving  produetivity,  effieieney  and  effectiveness  of  operations 
and  the  delivery  of  serviees  through  appropriate  and  effective  use  of  IT. 

Assess  IT  investments  and  progress  on  key  Air  Force  IT  initiatives  against 
performanee  goals. 

Information  Assurance: 

Monitor  information  protection  states  for  all  network/systems  within  the 
MAJCOM/HAF  Functional  area  and  partieipate  in  risk-based  evaluations  to  support 
operational  upgrades  or  operational  decisions. 

Ensure  new  systems  comply  with  security  architectures  and  are  properly 
certified/accredited  and  support  an  overall  security  plan. 

Provide  oversight  for  MAJCOM/Funetional  systems  reporting  processes  for 
Government  Information  Security  Reform  Act  (GISRA). 

Enforce  AF  CIO’s  vision  and  strategy  with  respect  to  information  assurance  to 
include  the  five  pillars  of  lA  (availability,  integrity,  confidentially,  authenticity,  and  non¬ 
repudiation). 

Capture  and  report  standardized  information  assurance  metrics  as  indicated  in  AF- 
CIO  policy  to  support  the  Clinger-Cohen  Act,  Government  Information  Security  Reform 
Act  (GISRA)  and  other  DoD  and  Federal  Government  requirements. 
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Ensure  IT  resource  decisions  include  robust  information  assurance  considerations 
and  are  integrated  with  the  HAF  and  MAJCOM  planning,  budgeting,  and  program 
management  processes. 

Ensure  personnel  receive  appropriate  information  security  training. 

Standards  &  Architecture: 

Facilitate  development,  and  provide  oversight  to  ensure  compliance  with  AF 
Enterprise  and  applicable  domain  architectures  and  IT  standards. 

Strategic  Planning: 

Advise/assist  the  AF-CIO  in  defining  strategic  direction,  deciding  issues,  and  in 
reviewing  proposed  policies,  methods  and  approaches. 

Develop  and  maintain  a  functional  area  or  MAJCOM  information  resources 
management  (IRM)  strategy  consistent  with  the  AF-ClO  direction  and  incorporate  the 
strategy  in  the  Functional  or  MAJCOM  strategic  plan. 

Help  develop,  mature,  and  implement  the  AF-CIO  Strategy. 

Include  information  management  public  law  issues  in  all  planning,  programming 
and  budgeting.  This  includes  records  management.  The  Privacy  Act,  The  Freedom  of 
Information  Act,  The  Paperwork  Reduction  Act,  The  Information  Technology 
Management  Reform  Aet,  Government  Paperwork  Elimination  Act  and  0MB  Circular 
A-130. 


Training  and  Education: 

Provide  oversight  of  the  IT  workforee  development  program. 

Provide  eontinuing  edueation  for  the  CIO  Community. 

Provide  oversight  of  IT  user  training. 

Information  and  Knowledge  Management: 

Promote  effeetive  information  sharing  and  management. 

Oversee  and  develop  strategie  guidanee  for  the  Records  Management  and 
Administrative  Communieations  Program. 

Oversee  and  develop  strategie  guidanee  for  the  MAJCOM  Data  Administration 
Program. 

Oversee  and  interpret  AF  guidanee  for  the  MAJCOM  Freedom  of  Information 
Aet  and  Privaey  Aet  Programs. 

Enforee  federal  statutory  and  regulatory  requirements  that  impact  information 
management. 

Process  Improvement: 

Develop  aetive  partnerships  with  mission  and  business  owners  seeking  to 
transform  their  operations  with  information  teehnology  infusions. 

Technology  Assessment: 

Advocate  state  of  the  art  teehnology  that  gives  us  the  competitive  edge  while 
balancing  the  technological  risk,  costs,  and  objectives  when  fielding  new  technologies. 
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E-Government/E-Business: 

Advocate  E-initiatives,  such  as  E-Commerce,  that  lead  to  more  effective  and 
efficient  mission/business  practices. 


131 


Appendix  D 


AF-CIO  Focus  Areas 

“Architecture:  Integrated  Operational,  Systems,  and  Technical  Architectures  or 
"views"  will  comprise  the  Air  Force  Information  Technology  Architecture  required 
by  the  Information  Technology  Management  Reform  Act/Clinger-Cohen  Act  of  1996. 
This  set  of  integrated  IT  architectures  will  help  the  Air  Force  CIO  guide  the  evolution 
and  maintenance  of  its  existing  information  technology  (IT)  systems,  and  the 
acquisition  of  new  IT  systems,  to  achieve  Air  Force  strategic  missions  and  goals  for 
the  21st  century. 

Business  Process  Reengineering:  The  fundamental  re-thinking  and  radical  redesign 
of  business  processes  to  achieve  dramatic  improvements  in  critical,  contemporary 
measures  of  performance,  such  as  cost,  quality,  service,  and  speed.  Seeks 
breakthroughs,  not  by  enhancing  existing  processes,  but  by  discarding  and  replacing 
them  with  entirely  new  ones.  The  ITMRA  require  processes  to  be  redesigned  and 
improved  before  the  aequisition  of  teehnology. 

Capital  Planning  and  Investment  Control:  The  CIO  shall  provide  assistance  to  the 
Seeretary  of  the  Air  Foree  and  other  senior  management  personnel  to  ensure  that 
information  teehnology  is  aequired  and  information  resources  are  managed  in  a 
manner  that  implements  the  ITMRA  and  the  PRA. 

Information  Resource  Management:  Information  Resource  Management  (IRM)  is 
a  management  funetion  dealing  with  effieient  management  of  information  and  data 
elements  throughout  their  lifeeyele.  IRM  eneompasses  the  planning,  budgeting,  and 
supervising  of  the  faeilities,  systems  and  organizations  associated  with  government 
information  in  accordanee  with  publie  laws  and  regulations.  It  covers  both  the 
information  itself  and  related  resourees,  sueh  as  personnel,  hardware,  software,  funds, 
and  technology.  The  Air  Foree's  IRM  program  supports  the  delivery  of  Air  Force 
programs  and  the  conduct  of  internal  management  functions  through  the 
administration  of  computer,  teleeommunieations  and  related  technologies  and 
management  of  forms,  reports,  and  automated  and  manual  information  systems. 

Information  Technology  Acquisition:  Section  5 124  of  the  ITMRA  restored  the  IT 
acquisition  authority  of  Federal  Executive  Agencies,  which  included  the  Military 
Services.  In  general,  this  authority  includes  the  following: 

1)  To  acquire  IT  as  authorized  by  law. 

2)  To  enter  into  a  contract  that  provides  for  multi  agency  acquisitions  of  IT  in 
accordance  with  guidance  issued  by  the  Director  of  0MB. 
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3)  If  the  0MB  Director  finds  that  it  would  be  advantageous  for  the  federal 
government  to  do  so,  to  enter  into  a  multi-agency  contract  for  procurement  of 
commercial  items  of  IT  that  requires  each  Executive  Agency  covered  by  the  contract, 
when  procuring  such  items,  either  to  procure  the  items  under  that  contract  or  to  justify 
an  alternative  procurement  of  the  items. 

Performance  Measures:  In  fulfilling  the  responsibilities  of  the  Information 
Technology  Management  Reform  Act/Clinger-Cohen  Act  1996  (ITMRA),  Paper 
Reduction  Act  (PRA),  and  the  Government  Performance  Results  Act  (GPRA),  the 
Secretary  of  the  Air  Force  establishes  goals  for  improving  the  efficiency  and 
effectiveness  of  Air  Force  operations.  The  Air  Force  CIO  advises  the  Secretary  on 
improving  the  effectiveness  of  Air  Force  operations  through  the  effective  use  of 
information  technology. 

Standards:  Conform  to  standards  outlined  in  the  JTA  and  DISA  Center  for 
Standards. 

Strategic  Plan:  The  Air  Force  Director  of  Communications  and  Information 
produced  an  Information  Technology  Management  Strategic  Plan  in  1997,  in  direct 
response  to  the  Clinger-Cohen  Act  1996.  A  new  plan  is  under  development  to  more 
closely  align  the  AF-CIO  with  Air  Force  strategic  planning.  The  AF-CIO  Strategic 
Plan,  is  intended  to  guide  development  of  more  detailed  plans  but  will  not  address 
specific  programs,  projects,  or  detailed  budgets.  Implementation  plans  will  pick  up 
where  the  strategic  plan  ends. 

Training  and  Education:  Training  and  educating  all  Air  Force  personnel  in  the 
concepts  and  tenets  of  Information  Resources  Management  IRM)  is  key  to  the 
successful  use  of  Information  Technology  (IT)  by  the  Air  Force.  Institutionalizing  the 
precept  of  information  as  an  Air  Force  strategic  resource  is  critical  to  the  future  of  the 
Air  Force.  The  CIO  is  charged  with  these  responsibilities.” 
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Appendix  E 


Clinger-Cohen  Core  Competencies 
Revised  September  2000 

The  Clinger-Cohen  Core  Competencies  have  been  endorsed  to  serve  as  a  baseline  to 
assist  government  agencies  in  complying  with  Section  5125(C)(3)  of  the  Clinger-Cohen 
Act.  To  perform  effectively  in  each  competency  area  below,  an  organization  should 
possess  the  knowledge,  skills  and  abilities  in  each  competency. 

1.0  Policy  and  Organizational 

1 . 1  Department/ Agency  missions,  organization,  functions,  policies,  procedures 

1.2  Governing  laws  and  regulations  (e.g.,  Clinger-Cohen,  GPRA,  PRA,  GPEA, 
0MB  Circular  A-130  ,  PDD  63) 

1.3  Federal  government  decision-making,  policy  making  process  and  budget 
formulation  and  execution  process 

1.4  Linkages  and  interrelationships  among  Agency  Heads,  COO,  CIO,  and  CFO 
functions 

1.5  Intergovernmental  programs,  policies,  and  processes 

1.6  Privacy  and  security 

1.7  Information  Management 

2.0  Leadership/Managerial 

2.1  Defining  roles,  skill  sets,  and  responsibilities  of  Senior  Officials,  CIO  staff 
and  stakeholders 

2.2  Methods  for  building  federal  IT  management  and  technical  staff  expertise 

2.3  Competency  testing  -  standards,  certification,  and  performance  assessment 

2.4  Partnership/team-building  techniques 

2.5  Personnel  performance  management  techniques 

2.6  Principles  and  practices  of  knowledge  management 

2.7  Practices  which  attract  and  retain  qualified  IT  personnel 

3.0  Process/Change  Management 

3.1  Techniques/models  of  organizational  development  and  change 

3.2  Techniques  and  models  of  process  management  and  control 

3.3  Modeling  and  simulation  tools  and  methods 

3.4  Quality  improvement  models  and  methods 

3.5  Business  process  redesign/reengineering  models  and  methods 
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4.0  Information  Resources  Strategy  and  Planning 

4.1  IT  baseline  assessment  analysis 

4.2  Interdepartmental,  inter-agency  IT  functional  analysis 

4.3  IT  planning  methodologies 

4.4  Contingency  planning 

4.5  Monitoring  and  evaluation  methods  and  techniques 
5.0  IT  Performance  Assessment:  Models  and  Methods 

5.1  GPRA  and  IT:  Measuring  the  business  value  of  IT,  and  customer  satisfaction 

5.2  Monitoring  and  measuring  new  system  development:  When  and  how  to  "pull 
the  plug"  on  systems 

5.3  Measuring  IT  success:  practical  and  impractical  approaches 

5.4  Processes  and  tools  for  creating,  administering,  and  analyzing  survey 
questionnaires 

5.5  Techniques  for  defining  and  selecting  effective  performance  measures 

5.6  Examples  of  and  criteria  for  performance  evaluation 

5.7  Managing  IT  reviews  and  oversight  processes 


6.0  Project/Program  Management 

6.1  Projeet  seope/requirements  management 

6.2  Projeet  integration  management 

6.3  Projeet  time/eost/performanee  management 

6.4  Projeet  quality  management 

6.5  Project  risk  management 

6.6  Project  procurement  management 

7.0  Capital  Planning  and  Investment  Assessment 

7.1  Best  practices 

7.2  Cost  benefit,  economic,  and  risk  analysis 

7.3  Risk  management-  models  and  methods 

7.4  Weighing  benefits  of  alternative  IT  investments 

7.5  Capital  investment  analysis-  models  and  methods 

7.6  Business  case  analysis 

7.7  Integrating  performance  with  mission  and  budget  process 

7.8  Investment  review  process 

7.9  Intergovernmental,  Federal,  State,  and  Local  Projects 
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8.0  Acquisition 

8.1  Alternative  functional  approaches  (necessity,  government,  IT)  analysis 

8.2  Alternative  acquisition  models 

8.3  Streamlined  acquisition  methodologies 

8.4  Post-award  IT  contract  management  models  and  methods,  including  past 
performance  evaluation 

8.5  IT  acquisition  best  practices 

9.0  E-Govemment/Electronic  Business/Electronic  Commerce 

9.1  Strategic  business  issues  &  changes  w/advent  of  E-Gov/EB/EC 

9.2  Web  development  strategies 

9.3  Industry  standards  and  practices  for  communications 

9.4  Channel  issues  (supply  chains) 

9.5  Dynamic  pricing 

9.6  Consumer/citizen  information  services 

9.7  Social  issues 

10.0  IT  security/information  assurance 

10.1  Fundamental  prineiples  and  best  praetiees  in  lA 

10.2  Threats  and  vulnerabilities  to  IT  systems 

10.3  Legal  and  poliey  issues  for  management  and  end  users 

10.4  Sourees  for  IT  seeurity  assistanee 

10.5  Standard  operating  proeedures  for  reaeting  to  intrusions/misuse  of  federal  IT 
systems 

11.0  Technical 

11.1  Information  technology  architectures,  client/server,  collaborative  processing, 
telecommunications 

1 1.2  Emerging/developing  technologies 

1 1.3  Information  delivery  technology  (internet,  intranet,  kiosks,  etc.) 

11.4  Software  development 

1 1.5  Data  management 

12.0  Desk  Top  Technology  Tools 
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Screenshots  of  the  web-based  survey. 

Sereenshots  of  the  original  e-mail  notification  sent  to  204  potential  respondents 


iSS  Selection  for  AF-CIO  Office  Sponsored  Research  -  Message  (Rich  Text) 


[  Eile  Edit  Viev^  Insert  Format  Tools  Actions  Table  Help 


;ijl;  ®  Paste  AIJ  Iterns  -  8 


0VPoply  I  ^PeplytoAIJ  I  VJPForwaid  i  ^  t-  i  T 


lO  Vou  replied  on  1/15/2002  13: 13.  Click  here  to  find  all  related  messages. 


Sent;  Mon  l/H/2002  16:16 


From:  AFCIO  Survey 


Selecbton  for  AF-CIO  Office  Sponsored  Research 


Sir/Ma'am, 

You  have  been  selected  to  participate  in  this  research  survey  because  of  your  CIO-related  experience  and  expertise  as  an  AF 
IT/IRM  leader.  This  research  is  sponsored  by  the  AF-CIO  Office,  Pentagon,  Washington  D.C.. 

Research  Study  Survey 


Evaluating  the  Outcomes  of  the  Clinger-Cohen  Act  (CCA)  of  1996  on  the  AF-CIO  Office  and  USAF  Operations. 

Thank  you  in  advance  for  taking  the  time  to  participate  in  this  research  survey.  Very  little  research  has  been  conducted  in  this 
evolving  area  so  your  participation  is  extremely  important  as  it  lends  credence  to  this  new/changing  field  and  research  topic. 

Please  be  sure  you're  using  MS  Internet  Explorer  as  your  web  browser;  users  who  have  Netscape  as  their  default  browser  could 
experience  some  problems. 

The  survey  consists  of  eight  short  pages  and  should  take  about  20-30  minutes  to  complete.  Please  click  on  this  hyperlink  to  begin 
the  survey  http://en.afil.edu/env/af-cio 

Results  of  the  survey  will  be  made  available  upon  request.  Please  email  your  request  to  AFCIO. survevrai aft  edu 
Your  participation  is  important  to  the  success  of  this  research  study  and  we  would  like  to  thank  you  again  for  your  support. 

Sincerely. 

AF-CIO  Survey  Research  Team 
School  of  Engineering  and  Management 
Air  Force  Institute  of  Technology 

isUMiij  '■  . 

Otaw  AutaShapes  -  .•'«  Q  O-  ^  IS  |  ^  ^  =  vrak  ^  ^  1 
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Screenshot  of  the  Introduction  page 


AF-CIO  Survey 


Research  Study  Survey 
on 

Evaluating  the  Outcomes  of  the  Clinger- Cohen  Act  (CCA)  of  1996  on  the  AE-CIO  OfiSce  and  TJSAE  Operations 

Thank  you  for  taking  the  time  to  participate  in  this  research  survey  niis  research  is  sponsored  by  the  AE-CIO,  Mr.  John  Gilligan.  You  have  been  selected  to 
participate  in  this  survey  because  of  your  CIO-related  experience  and  expertise  as  a  TJSAE  TT  /IRM  professional. 

The  survey  is  intended  to  examine  outcomes  of  the  CCA  on  the  TJSAE,  as  well  as  explore  normative  models  created  for  the  federal  agency  CIO  position  and  it's 
roles.  Scott  A.  Bernard,  Ph.D.,  Virginia  Polytechnic  Institute  and  State  University,  created  the  original  guide.  It  has  been  updated  to  reflect  the  changes  to  the 
Federal  CIO  Competencies  (2000),  and  uses  the  enhanced  Federal  CIO  Position  Evaluation  Method  (ECPEM)  recommended  by  Dr.  Bernard. 

This  research  uses  the  1996  CCA  as  the  legislative  source  for  what  roles  and  responsibilities  a  CIO  should  be  assigned  in  the  TJSAE.  Part  1  of  this  research 
explores  the  roles  and  outcomes  of  the  AE-CIO  o£6ce  on  the  USAF  andUSAE  operations  in  the  form  of  open-ended  questions.  Part  2  asks  questions  about  the 
Federal  CIO  Competencies  (2000),  And  finally.  Part  3  requests  your  responses  using  amatnx  of  CIO  Roles  to  CIO  Competencies, 

The  survey  consists  of  eight  short  pages  and  should  take  about  20-30  minutes  to  con^lete.  Please  complete  each  page  before  moving  on  (e.g,,  hitting  the 
CONTINTJE  button  at  the  bottom  of  the  page)  as  you  will  not  be  able  to  return  to  the  previous  page.  Shoiid  you  proceed  to  the  next  page  without  entenng  data, 
please  complete  the  survey,  then  return  to  the  begirming  of  the  survey  by  going  back  to  http^/en  afit.edu/env/af-cio/  and  completing  the  skipped  area(s). 

As  the  research  is  an  academic  exploration  of  the  results  of  the  CCA,  your  responses  are  non-attnbutional.  Every  effort  has  been  made  to  maintain  anonymity, 
names  will  not  be  associated  with  responses.  Results  of  the  survey  will  be  made  available  ipon  request.  Please  em^  your  request  to  AFCIO.survev@afit.edu 

Your  participation  is  important  to  the  success  of  this  research  study  and  ttie  research  team  would  like  to  fliank  you  again  for  your  support. 

Sincerely, 

AE-CIO  Survey  Research  Team 
School  of  Engineering  and  Management 
Air  Force  Institute  of  Technology 
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Screenshot  of  the  Background  page 


AF-CIO  Survey 

Background  Information 


1,  Position  Title 
2,.  Years  in  Position 
Years  in  Organization 


4„  Years  Experience  in  IT  /  IRM  | 


Screenshot  of  Part  la 


Part  la: 

Competencies 

As  a  result  of  the  CCA  and  the  AF-CIO  Office,  is  die  .  ,  . 

USAP  to<l,y  m  H..  are,  rf  ...  Ye.  No 

Al.  Policy  /  Organizational  C  C  j 

1 

. .  zJ 

A2.  Leadersbp  /  Managenal  C  O  j 

1 

A3.  Process  /  Change  Management  C  C  j 

b 

A4.  Ir£ormation  Resource  Strategy  and  Planning  C  C  j 

- a 

a 

A5.  Performance  Assessment,  Models  and  Methods  C  C  j 

a 

.  a 

A6.  Project  /  Program  Management  C  C  j 

a 

a 

A7.  Capital  Planning  and  Investment  Assessment  C  C  j 

a 

.  a 

AS.  Acquisition  C  C  | 

A9.  E-Govemment  /  Electronic  Business /Electronic  Commerce  C  C  j 

- 3 

^1 

AlO.  n  Security /Information  Assurance  r  C  j 

- 3 

a 

All,  Technical  C  t?  | 

1 

All.  Desktop  Technology  Tools  C  C  | 

a 

a 

Continue  | 

- 

ll  . ■  ■  ■  ■  ■  . - . '^^jf 

is 
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Screenshot  of  Part  lb 


Part  lb: 

Performance 

With  regard  to  perfonnanre/  results  based  management,  is  Ae 

USA!F  tliffeieiit  today  ill  the  ai  ea  of...  Yes  No 

Bl.  Establishing  goals  for  improving  the  efficiency  and  effectiveness  ofTJSAF  ^  ^ 

operations 

ij 

B2. 

- 3 

Delivering  services  throughthe  effective  use  ofinformahontechnology  O  C 

3  1 

B3.  Preparing  annual  reports,  included  m  the  budget  submission  to  Congress,  ^  ^ 

Mi 

on  the  progress  of  achieving  goals 

d 

B4.  Ensuring  performance  measurements  are  prescribed  for  IT  used  or  to  be  ^  p 

id  i 

acquired 

-ij  p 

B5.  Ensuring  performance  measurements  measure  how  well  IT  supports  ^  p 

- 

USAP  programs 

B6.  Quantitatively  benchmarking  TJSAF  process  performance  against  pubEc  ^  p 

d 

and  private  processes 

. 

BT.  Analysing  USAP  missions  and  revising  mission-related  and  administrative  p  p 

processes  as  appropriate  before  making  significant  IT  investments 

3  = 

B8.  Ensuring  the  information  security  policies,  procedures,  and  practices  of  the  P  p 

UJ 

USAP  are  adequate  i 

.  '1 

Continue  { 
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Screenshot  of  Part  Ic 


Part  Ic: 

OntcmHeig/OMtpttts 

J'..-:  s,y  i.'iO  t  ilHfie  :^rldiei-vd  dgrdi'iriiiii- cratc«iDi«-is 
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Screenshot  of  Part  Id 


Part  Id: 

CIO  Roles 


Roles 

Cfipital  Planning  and  Couti'dl 

Iiifoimation  Tetlmoldgy  Acqiij:silidu 
Perfonntmte  Mettsnres 
Infoiiaatidu  Assuiance 
Staudards  &  Arcliitectiue 
S'ti'ategic  Planning 
Ti  ainiug  andEdiicatidn 
Iivfdiination  and  Kndwledge  Managdment 
Ptdfdss  littpi-dvdinent 
Teflmology  Assessment 
E-  G  ovenuneiit/E-Busiiie  ss 


(Please  click  dii  die  liyiierlink  beldw  to  see  the  respdnsibilities  for  these  roles) 


Reference:  htt 


'/WWW  cio.hq.afjnil>'nri\'ate.''maicDm  cios  haf  Innc  cio  rep  r  r.doc 


Please  answer  the  following  questions; 


What  would  you  say  your  role(s)  is  (are)  in  terms  of  CIO 

Zl,  related  duties  at  your  level  (does  not  have  to  be  selected  from 
the  above  list)? 

22  ^.elating  to  the  question  above,  do  you  feel  these  roles  are 
"  appropriate  at  your  level?  Please  e^lain. 

Are  there  any  roles  you  would  include  or  eliminate  from  diose 
‘  you  are  currently  responsible  for? 

2^  Please  identify  any  shortfalls  as  it  effects  your  level  (e.g., 

'  staffing). 


Screenshot  of  Part  2 


Part  2: 

AF-CIO  Model 


This  section  briefly  describes  the  levels  of  Thompson’s  Oi^aniz&tional  Model  and  Federal  CIO  Competencies  from  the  Federal  CIO  Council.  Thompson  explains  the  followup  oi^aiusational  levels  as; 

Technical  Level.  “A  sub-organization  whose  problems  are  focused  around  ^(ecfiveper^rmance  of  the  techrdcai  function...  the  primary  [necessity]  are  those  imposed  by  the 
nature  of  the  technical  task.” 

Managerial  Level  ‘‘Services  the  technical  sub-orgfinization  by  mediatingwith  those  who  use  itsproducts,  and  procuring  the  resources  necessary  to  carry  outitsfonctions.” 

Institutional  Level  “A  wider  social  system  which  is  the  source  of  the  meaning  legitimization,  or  higher-level  support  which  makes  the  impiementanon  of  the  organization's  goals 
possible.” 

A  Please  rate  the  importance  you  would  give  to  each  of  the  following  CIO  competency  areas: 

(Select  one  “hiqiortance“  button  fi>r  each  conqtetency  area) 

B  Please  relate  the  CIO  Competency  Areas  to  a'levef’  of  the  USAF,  as  listed  below: 

(Select  one  “level”  button  fbr  each  con^etency  area) 


CIO  Conqietency  Area 

Very  Important 

Somewhat 

Important 

Not  Importair 

Technical  Manorial  Institutional 
Level  Level  Level 

ML  Policy  and  Oi^anizatonal 

r 

r 

r 

1  ^ 

r 

c 

M2.  Leadership  1  Managerial 

c 

r 

r 

1  r 

r 

r 

M3.  Process  i  Change  Management 

c 

r 

r 

1  r 

r 

c 

M4.  Information  Resources  Strategy  and  Plannir^ 

c 

r 

1  r 

r 

C: 

M5.  Perfonnance  Assessment:  Models  and  Methods 

c 

r 

r 

1  r 

r 

C: 

M6.  Project  /  Program  Man^iuent 

r 

r 

r 

1  r 

r 

C: 

M7.  Capital  Plannii^  and  Investment 

r 

c 

r. 

1 

c 

■n 

MS.  Acquisition 

r 

c 

r- 

1  c 

r 

r 

M9.  E-Govemment 

c 

c 

c 

1  ^ 

n 

Ci 

MIO.  IT  Security  /  Infbnnation  Assurance 

c 

c 

c 

r 

Ci 

r 

Mil.  Technical 

r 

o 

c 

1  ^ 

C: 

C: 

M12.  Desktop  Technology  Tools 

c 

c 

c 

n 

r 

r 

Competencies  scnuce:  Fedetel  CIO  Council,  2000. 

Organiational  Model  source:  James  Thompson,  Organisations  in  Action,  1967. 
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Screenshot  of  Part  3a 


Part  3a: 

Federal  CIO  Position  Evalnation  Method  (FCPEM) 

A.  Please  indicate  whether  each  mandate  is  a  “goal  for'  or  goal  or  the  organization.  (Goals  “for”  an  organization  are  eternally  induced.  Goals  “of'  an  organization 
are  mtemally  developed) 

(Use  ‘T'h  “0”,  or  in  the  box  below) 

B  Please  indicate  the  degree  of  complexity  for  each  mandate  (High^  Medium  or  Low). 

(Use  or  "L”  in  the  box  below) 

The  Evaluation  Standard  for  Each  CIO  Role 

Goal  for/of  Agency 
F  0  Both 

Complexity  ^ 
of  the  CTO 

Role  Ai'ea 

H  M  L 

El.  Was  a  CIO  position  formally  designated  and  established? 

r 

r 

r 

n/a 

E2.  Is  the  CIO  a  member  of  Executive  Level  IV? 

r 

~r 

r 

n/a 

E3.  Is  direct  ClO-agency  head  reporting  established  in  writing? 

r 

~C 

c 

n/a 

E4.  Does  the  designation  document  make  IRM  the  CIOs  principle  duty? 

r 

~C 

c 

n/a 

E5.  Does  the  CIO  facilitate  reviews  to  improve  IRM-related  processes,  including  reducing  the  public  information  collection 
burdens? 

r 

C: 

r 

C'  c  r 

E6  .  Is  there  a  CIO  &  CPO  facilitated  process  for  identifying  all  agency  program  IT  needs,  strategies,  systems,  capabilities? 

r 

~C 

c 

C'  c 

E7„  Does  the  CIO  facilitate  the  evaluation  of  information  collections  independent  of  CIO  programroles? 

r 

~r 

c 

p  p  p 

E8.  Does  the  CIO  facilitate  an  IT  Capital  Planning  process,  advise  agency  head/mgmt.  &  ensure  IT  IS  acquired  &IRM/n’A  ^  ^  ^ 

are  done  lAW  PRA'95  &  agency  head  priorities'? 

E9.  Does  the  CIO  facilitate  anITA  that  ties  to  C^italPlanningandfoUows  0MB  A-130/OMB  97-16format/’guidance?  C  C  C  O  C  C 

ElO., Does  the  CIO  review  IT  programs  for  <10%  variance  in  cost,  schedule,  performance  C  C  C  C  C 

Ell.Is  there  an  agency  IT  Strategic  Plan  and  is  It  reflected  in  die  FY  Strategic  Plan  and  the  Performance  Report?  C  C-  C  H  C 

Ell.Does  the  agency  have  a  CIO-facilitated  IT  Workforce  Plan  that  addresses  needed  IRM  skills,  training,  hiring,  &  ^  ^  p  p  P  p 

professional  development? 

E13.Does  the  CIO  report  in  writing  to  the  agency  head  each  year  on  how  IRM  capability  is  improving? 

C: 

c 

C 

c  c  r 

Continue  | 
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Screenshot  of  Part  3b 


Part  3h: 

Federal  CIO  Position  Evaluaiion  Method  (FCPEM) 


C.  This  page  relates  the  CIO  roles  from  the  Clmger- Cohen  Act  of  1996,  Section  5125,  with  the  Federal  CIO  Covincil  Competencies  2000  list. 

Please  indicate  the  related  CIO  competency  to  the  CIO  roles  per  the  CCA.  Please  use  the  number  related  to  each  competency.  You  may  use  more  than  one 
competency  for  each  role 

Federal  CIO  Council  Competencies  (2000) 

1  -  Pohcy  and  Organizational 

2  -  Leadersh^  /  Managerial 

2  -  Process  /  Change  Management 
4-  Information  Resources  Strategy  and  Planning 

5  -  Performance  Assessment  Models  and  Methods 

6  -  Project  /  Program  Man^ement 

7  -  Coital  Planning  and  Investment  Assessment 

8  -  Acqmsition 

9  -  E-Govemment  /  E-Business  /  E-Commerce 

10  -  IT  Security  /  Information  Assurance 

11  -  Technical 

12  -  Desktop  Technology  Tools 


CIO  Roles 

From  tlie  CCA.  Section  5125 


CIO  Competencies 

From  the  Federal  CTO  CoimcU.  2000 
(Please  check  all  that  apply) 


CIO  Role  Evaluation  Standard 

1 

3 

4 

5 

6 

8 

9 

10 

11 

12 

FI.  The  CIO  facilitates  revievi's  to  improve  IRM-relatcd  processes, 
including  reducing  the  public  information  collection  burdens 

r 

IT 

r 

r 

r 

n 

n 

n 

n 

r 

r 

n 

F2.  A  CIO  &  CFO  facilitated  process  identifies  all  agency  program  IT 

n 

needs,  strategies,  systems,  and  c salabilities 

F2.  The  CIO  facilitates  the  evaluation  of  information  collections  independent 
of  CIO  program  roles 

r 

r 

n 

r 

r 

r 

r 

r 

r 

n 

n 

F4.  The  CIO  facilitates  an  IT  Capital  Planning  process,  advises  agency 
head/mgmt,  &  ensures  IT  is  acquired  &IRM/TrAis  done  IAWPRA'95 

c 

r 

c 

c 

c 

c 

c 

c 

c 

c 

r 

n 

&  asencv  head  onorities 

F5„  The  CIO  facilitates  an  ITA  that  ties  to  Capital  Planning  and  follows 

0MB  A-130/OMB  97-16  formsrt/guidstficc 

c 

n 

r 

n 

c 

n 

n 

n 

o 

r 

n 

n 

F6.  The  CIO  reviews  IT  programs  for  <10%  variance  in  cost,  schedule, 
and  performance 

□ 

n 

n 

n 

n 

□ 

□ 

n 

n 

r 

r 

r 

F7.  The  agency  IT  Strategic  Plan  is  refiected  in  the  FY  Strategic  Plan  and 
the  Performance  Report 

c 

r 

r 

r 

n 

n 

□ 

□ 

D 

c 

r 

n 

F8„  The  agency  has  a  ClO-facilitated  IT  Woritforce  Plan  tiiat  addresses 
needed  IRM  skills,  training,  hiring,  ^-professional  development 

n 

n 

r 

n 

n 

r 

n 

n 

n 

r 

r 

r 

F9,  The  CIO  reports  in  writing  to  the  agency  head  each  year  on  how  IRM 

n 

n 

r 

r 

r 

r 

n 

r 

r 

r 

r 

r 

Do  you  have  ai^  additional  comments  you  would  like  to  add? 
Please  write  any  comments  you  have  below. 


The  survey  is  now  complete)  Again,  thank  you  for  taking  the  time  to 
con^lete  this  survey.  Your  responses  are  estremely  valuable  to  this  research. 
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Appendix  G 


Table  38:  Data  from  web-based  survey,  Part  la 


Part  la,  Impact  from 

Core  Competencies 

#of 

Comments 

Part  la,  CCA  Impact  from 

Core  Competencies 

#of 

Comments 

Standardized  Approach 

19 

Better  Working  Relationship  with  Functionals 

2 

Working  Groups  at  Base/MAJCOM  Level 

15 

BPR  is  Practiced 

2 

Centralized  Management  of  Networks 

12 

Centralized  IT  Project  Selection 

2 

Responsive  Organizational  Structure 

11 

Certification  Processes 

2 

IT  Enterprise  Policy  Awareness 

10 

DODCERT  Effective 

2 

Business  Case  Development 

8 

Equipment  and  CC  Better 

2 

Information  as  a  Strategic  Resource 

8 

Greater  Level  of  Empowerment 

2 

CIO  Mandated  by  Law 

7 

No  Budgetary  Authority 

2 

Consolidation  of  Neworks/Servers 

7 

SPO  Overhead  too  Costly 

2 

Development  of  TCO 

7 

Vision  is  Unclear 

2 

More  Clear  Direction 

7 

Application  Improvements 

1 

Network  Centric  Awareness 

6 

Better  Visibility  into  IT  Spending 

1 

IT  Initiatives 

5 

Customer  Admin  fools 

1 

Strategic  Information  Approach 

5 

Greater  Interopability 

1 

Architecture  Office  Established 

4 

Higher  Level  Better  Involved 

1 

Better  Performance  Measures 

4 

Manpower  Shortages  Apparent 

1 

Config  Control  AF-wide 

4 

Non-standardized  Approach 

1 

Enterprise  Strategy 

4 

Oversight  Council 

1 

Network  Consolidation 

4 

PPBS  Hasn't  Changed  Enough 

1 

Timely  Acquisition 

4 

Reuse  Emphasis 

1 

AFCERT  Effective 

3 

Roles  are  Still  Unclear 

1 

Certificates  of  Networthiness 

3 

Technology  More  Available  to  Users 

1 

Industry  Best  Practices 

3 

Too  Slow  to  Catch  On 

1 

Too  Focused  on  Technology 

3 

Untimely  Acquisition  Process 

1 
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Table  39:  Data  from  web-based  survey,  Part  lb 


Part  lb,  Impact  from 

CCA,  Section  5123 

#of 

Comments 

Part  lb.  Impact  from 

CCA,  Section  5123 

#of 

Comments 

Use  of  Business  Models 

9 

Architecture  Office 

1 

IT  Initiatives 

8 

Better  Technical  Guidance 

1 

Too  Soon  to  Tell 

8 

Budget  Doesn't  Reflect  Actual  Purchases 

1 

High  Level  Support 

7 

Capability  Based  Metrics 

1 

Working  Groups  at  Base/M AJCOM  Level 

5 

Capability  Too  Slow 

1 

Enterprise  Solution 

4 

Cost  T racking  is  Better 

1 

IT  Enterprise  Policy  Awareness 

4 

Difficult  to  Measure 

1 

Little  Change  to  PPBS 

4 

DODcert  is  Effective 

1 

Accountability  Greater 

3 

Focus  Change  to  Business  Entity 

1 

CIO  Leadership 

3 

Force  Multiplier 

1 

Equipment  and  Config  Control  Better 

3 

Funding  Tied  to  performance 

1 

Responsive  Org  Structure 

3 

Integration  Better 

1 

Strategic  Planning 

3 

Lack  of  Funding  for  Missions 

1 

TCO 

3 

Lack  of  Process  for  Budget  Planning 

1 

AF  Portal 

2 

Metrics  Poor  Between  IT  and  Ops 

1 

AFCERT  is  Effective 

2 

More  Clear  Direction 

1 

BPR  has  Improved 

2 

Multiple  Channels  of  Connectivity 

1 

Goals  Tied  to  Money  and  Staffing 

2 

Network  Centric 

1 

Information  Collection 

2 

New  Reporting  Process 

1 

IT  Treated  as  Weapon  System 

2 

Leadership  not  Responsive 

1 

Low  Visibility  in  Reporting 

2 

Outcome  Measures 

1 

Metrics  Measure  Activity,  not  Performance 

2 

Performance  Reporting  Improvement  Slow 

1 

Network  Consolidation 

2 

Policy  Implementation  Poor 

1 

NOSC  is  Key 

2 

QOS  Good  Measures 

1 

Not  Possible  to  Measure  Performance 

2 

ROl  Tracking 

1 

Performance  Metrics  Good 

2 

Roles  are  Still  Unclear 

1 

Policies  not  Joint 

2 

Set  the  Standard  for  DoD  in  security 

1 

Politics  Hinder  Reporting 

2 

Slow  Implementation 

1 

Process  Not  Understood 

2 

Stale  Metrics 

1 

Realistic  to  IT  not  Ops 

2 

Standardization 

1 

Requirements  not  certain 

2 

Tied  budget  to  Expenditure  Reporting 

1 

TQM  Culture  Hinders 

2 

Training  Suffers 

1 

Vision  is  Unclear 

2 
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Table  40:  Data  from  web-based  survey,  Part  Ic 


Part  Ic,  Impacts  from  Key 
IRM  Achievement  Areas 

#of 

Comments 

Part  Ic,  Impact  from  Key 
IRM  Achievement  Areas 

#of 

Comments 

Realistic  to  IT,  Not  Ops 

25 

Business  Model 

1 

Too  Soon  to  Tell 

24 

Certificates  of  Networthiness 

1 

Process  Not  Understood 

19 

Certification  of  Personnel 

1 

High  Level  Support 

16 

CIO  Effective 

1 

TQM  Culture  Hinders 

15 

CIO  office  not  Effective 

1 

Goals  tied  to  Money  and  Staffing 

12 

Contracting  Support  Better 

1 

Perfomance  Oriented 

12 

Contrating  Efforts  Better 

1 

Little  Change  to  PPBS 

11 

Cost  Reporting  not  Understood 

1 

Using  Business  Models 

10 

Costs  not  Itemized 

1 

Slow  Implementation 

9 

Critical  Systems  Support 

1 

Network  Centric  Ops  Support 

9 

Data  Standards 

1 

Stale  Metrics 

5 

Decreased  Budget 

1 

Enterprise  Approach 

5 

Education  Needed 

1 

NOSC  is  Key 

5 

Improved  Bandwidth 

1 

Responsive  Org  Structure 

5 

Improved  Tech  Refresh 

1 

No  Money 

4 

IT  as  a  Weapons  System 

1 

Not  Reactive 

4 

Leadership  Not  Reactive 

1 

AF  Portal 

4 

Legacy  too  Costly 

1 

IT  Initiatives 

4 

Leveraged  Technology 

1 

Stovepiped 

3 

MAJCOM/Base  Working  Groups 

1 

CIO  responsibilty 

2 

MAJCOMs  Now  Lead 

1 

CITS  improved  Planning 

2 

Need  BPR 

1 

No  IT  Priorities 

2 

No  Authority 

1 

Not  CIO  Job 

2 

No  Staffing 

1 

Politics  with  Functionals 

2 

Reuse 

1 

Problems  Integrating  with  Ops 

2 

Robust  Networks 

1 

Standardization 

2 

Sharing  Info  NSS 

1 

Strategic  Planning 

2 

Short  of  Staff 

1 

Activity  Based,  not  Performance 

1 

System  Security  Reviews 

1 

Architecture  Standards 

1 

TCO  Accountability 

1 

BP  not  Standardized 

1 

Theater  Deployables 

1 

Budget  not  Requests  Successful 

1 
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